-- Logs begin at Tue 2022-08-30 11:08:35 UTC, end at Tue 2022-08-30 11:50:44 UTC. -- Aug 30 11:35:11.588939 dv-cinder-852208-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 11:35:11.597698 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 11:35:11.600427 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 11:35:11 2022] *** Aug 30 11:35:11.600628 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 11:35:11.600769 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 11:35:11.600908 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: nodename: dv-cinder-852208-12-iscsi Aug 30 11:35:11.601037 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: machine: x86_64 Aug 30 11:35:11.601174 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: clock source: unix Aug 30 11:35:11.601307 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: pcre jit disabled Aug 30 11:35:11.601445 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: detected number of CPU cores: 4 Aug 30 11:35:11.601577 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: current working directory: / Aug 30 11:35:11.601709 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: detected binary path: /usr/bin/uwsgi-core Aug 30 11:35:11.601848 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: your processes number limit is 47680 Aug 30 11:35:11.601981 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: your memory page size is 4096 bytes Aug 30 11:35:11.602117 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: detected max file descriptor number: 2048 Aug 30 11:35:11.602256 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: lock engine: pthread robust mutexes Aug 30 11:35:11.602551 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: thunder lock: enabled Aug 30 11:35:11.602871 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 11:35:11.603078 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 11:35:11.642222 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: Python main interpreter initialized at 0x55c6467369c0 Aug 30 11:35:11.642222 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: python threads support enabled Aug 30 11:35:11.642222 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: your server socket listen backlog is limited to 100 connections Aug 30 11:35:11.642222 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: your mercy for graceful operations on workers is 90 seconds Aug 30 11:35:11.642615 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 11:35:11.642673 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: *** Operational MODE: preforking *** Aug 30 11:35:11.642970 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: *** uWSGI is running in multiple interpreter mode *** Aug 30 11:35:11.642970 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: spawned uWSGI master process (pid: 87963) Aug 30 11:35:11.643234 dv-cinder-852208-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 11:35:11.649577 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: spawned uWSGI worker 1 (pid: 87979, cores: 1) Aug 30 11:35:11.649577 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: spawned uWSGI worker 2 (pid: 87980, cores: 1) Aug 30 11:35:11.649577 dv-cinder-852208-12-iscsi devstack@keystone.service[87963]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 11:35:15.830127 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 11:35:15.830127 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: from cryptography.utils import int_from_bytes Aug 30 11:35:15.830956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 11:35:15.830956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: from cryptography.utils import int_from_bytes Aug 30 11:35:15.835360 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 11:35:15.835360 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: from cryptography.utils import int_from_bytes Aug 30 11:35:15.835874 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 11:35:15.835874 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: from cryptography.utils import int_from_bytes Aug 30 11:35:15.880250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 11:35:15.880442 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 11:35:15.880578 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] command line args: [] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 11:35:15.880862 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 11:35:15.880983 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ================================================================================ {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 11:35:15.881266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] admin_token = **** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.881532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] config_dir = [] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.881693 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.881905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] config_source = [] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.882024 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 11:35:15.882102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] debug = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.882177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 11:35:15.882240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.883699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] command line args: [] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 11:35:15.883699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 11:35:15.883826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.883895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ================================================================================ {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 11:35:15.883956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] admin_token = **** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884083 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884142 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] config_dir = [] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884205 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884271 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] list_limit = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884395 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] config_source = [] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] log_config_append = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884516 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] debug = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884572 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884639 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] log_dir = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] log_file = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.884969 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885164 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885228 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885286 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] list_limit = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] log_config_append = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885479 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] log_dir = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885735 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] log_file = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.885994 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.886060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.886124 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.886182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.886361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.886437 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.886599 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.886724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.886906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.886998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.887075 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.887212 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.887340 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.887443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.887526 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.887644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.887778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.887873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.887957 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] publish_errors = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.888107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.888229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.888308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.888392 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.888560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.888692 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.888780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.888857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889237 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] standard_threads = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] publish_errors = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889628 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889911 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.889987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] use_journal = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890070 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] use_json = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890293 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] use_stderr = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890470 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890593 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] use_syslog = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] standard_threads = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890818 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.890965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.891147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.891270 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.891376 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.891461 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.891543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] use_journal = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.891617 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.891743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] use_json = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.891743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] use_stderr = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.891971 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.892093 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] use_syslog = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.892207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.892319 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 11:35:15.892438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.892552 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.892676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.892790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.892903 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.893022 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.893145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.893259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.893383 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.893497 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.893609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.893720 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.893838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.893959 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.894083 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.894197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.894309 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.894421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.894546 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.894661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.894781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.894900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.895020 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.895151 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.895266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.895390 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.895516 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.895630 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.895748 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.895861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.895974 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.896086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.896369 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.896490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.896603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.896715 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.896829 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.896996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.897206 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.897325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.897438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.897557 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.897676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.897798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.897912 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.898025 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.898147 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.898260 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.898371 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.898492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.898612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.898731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.898845 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.898964 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.899077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.899235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.899352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.899464 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.899576 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.899694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.899807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.899929 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.900049 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.900164 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.900276 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.900395 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.900514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.900636 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.900750 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.900868 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.900981 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.901094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.901206 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.901334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.901448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.901560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.901678 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.901800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.901913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.902104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.902226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.902344 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.902456 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.902567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.902685 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.902811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.902924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903240 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903334 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903546 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903852 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.903958 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.904070 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.904187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.904295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.904408 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.904527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.904630 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.904750 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.904863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.904964 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.905112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.905225 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.905346 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.905461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.905573 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.905685 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.905797 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.905921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.906036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.906147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.906260 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.906373 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.906477 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.906588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.906708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.906823 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.906947 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.907098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.907284 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.907398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.907526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.907643 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.907768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.907884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.907997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.908110 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.908223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.908336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.908460 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.908576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.908689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.908809 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.908924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.909036 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.909149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.909268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.909382 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.909495 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.909609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.909721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.909841 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.909955 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.910075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.910190 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.910309 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.910430 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.910552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.910674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.910789 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.910901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.911014 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.911142 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.911270 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.911386 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.911500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.911613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.911728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.911847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.911969 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.912074 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.912185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.912298 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.912411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.912526 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.912647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.912763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.912876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.912996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.913110 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.913223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.913417 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.913535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.913648 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.913760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.913880 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.914004 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.914127 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.914249 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.914370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.914484 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.914597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.914710 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.914823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.914944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.915059 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.915188 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.915304 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.915417 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.915530 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.915656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.915772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.915886 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.915999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.916112 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.916223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.916356 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.916472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] auth.external = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.916585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.916704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.916819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.916819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.917070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.917231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.917357 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.917357 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] auth.password = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.917573 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.917687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] auth.token = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.917801 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.917915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.918039 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] auth.external = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.918155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.918270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.918384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.918490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.918588 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.918692 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.918790 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.918904 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] auth.password = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.919008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.919168 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] auth.token = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.919286 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.919414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.919531 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] credential.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.919645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.919759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.919873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.919987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.920116 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.920231 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.920329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.920424 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.920519 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.920615 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.920720 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.920818 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.920914 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] credential.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921010 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921105 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921202 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921595 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921785 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921889 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.921986 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922181 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922276 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922371 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922475 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922572 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922669 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922764 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] federation.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922859 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.922954 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923058 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923300 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923401 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923480 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923553 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923634 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923782 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.923927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] federation.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924007 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924083 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924228 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924373 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924529 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924673 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924899 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.924974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925050 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925124 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925269 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925610 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925696 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.925928 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926003 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926160 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926388 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926693 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926829 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.926999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.927108 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.927256 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.927406 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.927533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.927614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.927710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.927864 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.928032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.928117 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.928204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.928281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.928437 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.928555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.928645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.928821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.928947 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.929031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.929120 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.929299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.929425 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.929507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.929582 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.929749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.929902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.929988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.930064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.930226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.930352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.930433 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.930521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.930670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.930794 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.930877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.930953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.931206 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.931318 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.931398 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.931472 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.931589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.931671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.931807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.931927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.932078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.932196 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.932310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.932432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.932557 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.932676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.932791 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.932905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.933019 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.933133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.933258 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.933377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.933490 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.933604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.933719 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.933833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.933966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.934084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.934199 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.934313 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.934426 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.934540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.934662 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.934779 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.934894 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.935008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.935186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.935306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.935434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.935552 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.935667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.935781 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.935896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.936009 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.936132 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.936249 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.936362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.936476 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.936590 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.936705 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.936828 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.936945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.937066 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.937182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.937331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.937448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.937571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.937693 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.937808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.937931 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.938053 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.938169 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.938292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.938409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.938523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.938636 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.938750 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.938864 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.938988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.939104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.939286 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.939403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.939518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.939631 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.939754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.939871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.939985 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.940100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.940215 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.940329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.940452 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.940574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.940689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.940803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.940916 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.941031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.941155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.941270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.941421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.941536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.941649 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.941763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.941918 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.942038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.942153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.942268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.942381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.942496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.942620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.942736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.942850 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.942964 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.943078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.943208 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.943336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.943460 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.943576 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.943690 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.943690 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.943905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.944029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.944145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.944260 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.944374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.944488 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.944602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.944724 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.944840 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] resource.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.944954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.945068 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.945182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.945297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.945420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.945536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.945650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.945763 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.945878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.945992 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.946115 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.946231 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.946346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] resource.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.946459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.946575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.946701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.946817 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.946931 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.947045 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.947172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] role.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.947288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.947515 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] role.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.947641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.947756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.947872 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.947986 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.948100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.948222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.948346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.948461 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.948576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.948689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.948804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] role.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.948926 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.949042 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] role.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.949156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.949270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.949384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.949499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.949620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.949736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.949851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.949965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.950079 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.950200 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.950324 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.950441 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.950560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.950674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.950788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.950903 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.950903 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.951159 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.951278 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.951392 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.951506 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.951621 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.951750 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.951867 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.951982 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.952096 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.952210 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.952324 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.952324 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.952550 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.952665 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.952780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.952893 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.953016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.953132 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.953247 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.953361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.953784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.953904 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.954027 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.954143 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.954258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.954372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.954487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.954601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.954725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.954841 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.954956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.955069 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.955200 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.955316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.955443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.955560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.955673 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.955789 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] token.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.955904 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.956017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.956140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.956256 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.956371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.956485 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.956600 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.956714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.956838 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.956954 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.957069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.957184 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.957299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] token.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.957413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.957575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.957695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.957809 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.957924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.958040 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.958155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.958277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.958393 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.958508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.958622 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.958735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.958849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.958973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.959088 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.959228 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.959344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.959472 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.959662 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.959784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.959784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.960012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.960126 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.960240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.connection = **** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.960354 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.960468 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.960596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.960713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.960828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.960951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.961078 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.961194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.961194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.961410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.961534 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.961658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.connection = **** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.961774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.961888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.962009 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.962125 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.962248 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.962364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.962479 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.962594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.962708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.962823 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.962946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.963062 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.963312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.963430 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.963545 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.963659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.963787 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.963904 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964019 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87979) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 11:35:15.964833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 11:35:15.964833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 11:35:15.991860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:15.992087 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:15.992309 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:15.992624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:15.992858 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:15.993082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.000488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.001325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.028008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.028154 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.028703 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.028932 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.080613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.080762 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.128318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.128546 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.128670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.128835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.282500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.282801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.282971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.283058 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.283182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.283265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.283329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.283453 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.283533 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.283662 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.283744 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.283851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.306443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.306576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.306735 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.306833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.306983 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.307064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.307147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.307218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.307391 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.307518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.307637 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.307720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.307784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.307873 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.318640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.318855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.331656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.331790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.331929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332027 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332222 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332286 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332407 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332693 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.332900 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.333131 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87979) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 11:35:16.349297 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.349505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 11:35:16.349834 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.350035 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.350161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 11:35:16.350290 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 11:35:16.350374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.350588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.350712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.350854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 11:35:16.350949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.351070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.351316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.351440 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.351579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.351817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.360544 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.360825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.361094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.361219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.361359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.361462 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.361717 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.361996 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.368235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.368904 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.371073 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.371384 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.371659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.371783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.371931 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.372059 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.372210 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.372298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.372453 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.372579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.372718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.372815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.372935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.373052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.373334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.373462 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.373614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.373699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.373849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.373975 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.374097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.374376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.395891 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.396618 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.397346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.397626 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.398716 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.399152 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.402260 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.402542 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.402820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.403769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.404051 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.404328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.409062 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.409346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.410700 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.410987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.413495 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.413781 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.414064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.414341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.414613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.414920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.415045 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.415231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.415315 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.415473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.415619 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.415803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.416078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.418436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.418752 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.419039 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.432463 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.432738 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.434388 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.434667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.437568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.438481 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.439958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.439958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.441561 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.441845 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.444208 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.446160 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.447293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.449262 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.450452 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.450717 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.451030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.451338 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.451629 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.451911 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.452316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.452591 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.452896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.453177 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.453465 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.454142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.467740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.468027 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.468294 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.468564 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.469720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.470007 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.470272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.470529 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.475362 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.476886 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.477208 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.478392 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.478671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.478809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.480380 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.480654 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.481290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.481593 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.481978 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.482267 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.482567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.483278 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.483546 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.483838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.484119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.484400 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.493681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.493959 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.494327 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.494606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.494885 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.495195 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.495485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.495662 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.495846 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.496116 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.496398 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.496671 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.496948 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.497241 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.510557 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.512395 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.513445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.515394 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.516302 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.517725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.518260 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.519330 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.519624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.519768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.519910 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.521143 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.521434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.521723 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.526656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.528198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.528630 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.530123 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.531156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.531443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.531720 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.531994 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.532942 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.533250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.533522 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.533800 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.539695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.539985 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.540258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.541638 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.541924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.542194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.547005 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.547315 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.547589 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.547859 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.548125 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.548405 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.548679 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.548920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.549045 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87979) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 11:35:16.549184 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.549283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.549479 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.549665 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.549804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.549972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.550093 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.550211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.550329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.550446 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.550576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.550696 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.550826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.551066 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.551362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.551613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.551871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.552128 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.552407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87979) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 11:35:16.584213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c6467369c0 pid: 87980 (default app) Aug 30 11:35:16.586078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c6467369c0 pid: 87979 (default app) Aug 30 11:35:16.587551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a32fba-4062-4605-84f2-eb1c07852f24 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:16.587832 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a32fba-4062-4605-84f2-eb1c07852f24 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:16.588094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a32fba-4062-4605-84f2-eb1c07852f24 None None] PATH_INFO: `/v3` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:16.592326 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Tue Aug 30 11:35:16 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:23.133569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66dab5-ea55-476f-ac90-8636ce4594ba None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:23.134153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66dab5-ea55-476f-ac90-8636ce4594ba None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:23.134153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66dab5-ea55-476f-ac90-8636ce4594ba None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:23.138466 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:35:23.143904 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:23.144476 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:23.144476 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:23.218360 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: INFO dbcounter [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] Registered counter for database keystone Aug 30 11:35:23.239244 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG oslo_db.sqlalchemy.engines [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87980) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 11:35:23.247172 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writer thread running {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 11:35:23.429497 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG passlib.handlers.bcrypt [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87980) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 11:35:23.429792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG passlib.handlers.bcrypt [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87980) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 11:35:23.490833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:23.493534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:23.644409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 11:35:24.154319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66dab5-ea55-476f-ac90-8636ce4594ba None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:24.233059 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: INFO dbcounter [None req-6c66dab5-ea55-476f-ac90-8636ce4594ba None None] Registered counter for database keystone Aug 30 11:35:24.257532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG oslo_db.sqlalchemy.engines [None req-6c66dab5-ea55-476f-ac90-8636ce4594ba None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87979) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 11:35:24.265514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writer thread running {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 11:35:24.385638 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66dab5-ea55-476f-ac90-8636ce4594ba None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:24.391153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d44d3-ca82-4ec6-80cf-a8b892c93dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:24.391785 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d44d3-ca82-4ec6-80cf-a8b892c93dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:24.392060 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d44d3-ca82-4ec6-80cf-a8b892c93dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:24.392328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d44d3-ca82-4ec6-80cf-a8b892c93dc2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:24.401454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d44d3-ca82-4ec6-80cf-a8b892c93dc2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:24.452485 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b97d44d3-ca82-4ec6-80cf-a8b892c93dc2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 11:35:24.452827 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d44d3-ca82-4ec6-80cf-a8b892c93dc2 admin admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:24.463193 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Tue Aug 30 11:35:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 11:35:24.470069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:24.487011 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432fc0-2ec6-46b1-90b3-3f3a7f0a42b5 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:24.493519 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f7b02-d4be-4737-95c9-84d85940f5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:24.494711 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f7b02-d4be-4737-95c9-84d85940f5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:24.494987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f7b02-d4be-4737-95c9-84d85940f5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:24.495282 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f7b02-d4be-4737-95c9-84d85940f5d8 admin admin] PATH_INFO: `/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:24.497094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f7b02-d4be-4737-95c9-84d85940f5d8 admin admin] RBAC: Authorizing `identity:get_project(project_id=a2579099690b4b9ea0cc6cb7695b8b5a)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:24.550777 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8a0f7b02-d4be-4737-95c9-84d85940f5d8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 11:35:24.551135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f7b02-d4be-4737-95c9-84d85940f5d8 admin admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:24.555868 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Tue Aug 30 11:35:24 2022] GET /identity/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:26.402106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4649c7-f5b6-45d0-b144-63f76aecbc46 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:26.402991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4649c7-f5b6-45d0-b144-63f76aecbc46 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:26.402991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4649c7-f5b6-45d0-b144-63f76aecbc46 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:26.409787 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:26.418255 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1ea96-96d2-43ca-bc87-b2eae188ba21 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:26.418707 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1ea96-96d2-43ca-bc87-b2eae188ba21 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:26.418800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1ea96-96d2-43ca-bc87-b2eae188ba21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:26.439836 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-49b1ea96-96d2-43ca-bc87-b2eae188ba21 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:26.441345 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-49b1ea96-96d2-43ca-bc87-b2eae188ba21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:26.449915 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 30 11:35:26.714716 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4649c7-f5b6-45d0-b144-63f76aecbc46 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:26.733247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4649c7-f5b6-45d0-b144-63f76aecbc46 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:26.740544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769831a-f88f-42e8-9710-877777c8b47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:26.741476 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769831a-f88f-42e8-9710-877777c8b47e admin admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:26.741879 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769831a-f88f-42e8-9710-877777c8b47e admin admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:26.742250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769831a-f88f-42e8-9710-877777c8b47e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:26.755089 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769831a-f88f-42e8-9710-877777c8b47e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:26.757500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769831a-f88f-42e8-9710-877777c8b47e admin admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:26.767030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Tue Aug 30 11:35:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 11:35:26.773028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1ea96-96d2-43ca-bc87-b2eae188ba21 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:26.787784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1ea96-96d2-43ca-bc87-b2eae188ba21 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:26.793747 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32cf01-065d-4d52-8f61-2b8f03cbaa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:26.794372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32cf01-065d-4d52-8f61-2b8f03cbaa4b admin admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:26.794645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32cf01-065d-4d52-8f61-2b8f03cbaa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:26.794905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32cf01-065d-4d52-8f61-2b8f03cbaa4b admin admin] PATH_INFO: `/v3/users/506323730e1e4e22bdf37b533b4f458b` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:26.796482 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32cf01-065d-4d52-8f61-2b8f03cbaa4b admin admin] RBAC: Authorizing `identity:get_user(user_id=506323730e1e4e22bdf37b533b4f458b)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:26.798330 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32cf01-065d-4d52-8f61-2b8f03cbaa4b admin admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:26.799951 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Tue Aug 30 11:35:26 2022] GET /identity/v3/users/506323730e1e4e22bdf37b533b4f458b => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:28.745501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa53605-f6d0-40bc-9099-50e08a3984a0 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:28.746054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa53605-f6d0-40bc-9099-50e08a3984a0 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:28.746054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa53605-f6d0-40bc-9099-50e08a3984a0 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:28.747391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:28.752778 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3833f31-ac69-4e44-b194-38ca1560601e None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:28.753205 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3833f31-ac69-4e44-b194-38ca1560601e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:28.753296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3833f31-ac69-4e44-b194-38ca1560601e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:28.767626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3b72-59ed-4d99-9544-ff80a5794191 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:28.767882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3b72-59ed-4d99-9544-ff80a5794191 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:28.768202 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3b72-59ed-4d99-9544-ff80a5794191 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:28.769517 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:28.772354 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-f3833f31-ac69-4e44-b194-38ca1560601e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:28.773647 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f3833f31-ac69-4e44-b194-38ca1560601e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:28.774336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124ff9-be4d-4147-bdbb-f7cc2c94c7b5 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:28.774647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124ff9-be4d-4147-bdbb-f7cc2c94c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:28.774926 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124ff9-be4d-4147-bdbb-f7cc2c94c7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:28.817134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG passlib.handlers.bcrypt [None req-a1124ff9-be4d-4147-bdbb-f7cc2c94c7b5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87979) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 11:35:28.817526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG passlib.handlers.bcrypt [None req-a1124ff9-be4d-4147-bdbb-f7cc2c94c7b5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87979) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 11:35:28.854059 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:28.866866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-a1124ff9-be4d-4147-bdbb-f7cc2c94c7b5 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:28.868501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-a1124ff9-be4d-4147-bdbb-f7cc2c94c7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:28.970334 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:29.116254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3833f31-ac69-4e44-b194-38ca1560601e None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:29.140704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:29.223383 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3833f31-ac69-4e44-b194-38ca1560601e None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:29.240211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1124ff9-be4d-4147-bdbb-f7cc2c94c7b5 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:29.259313 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647859e-f318-437a-9864-30a53570d86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:29.259945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647859e-f318-437a-9864-30a53570d86f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:29.260223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647859e-f318-437a-9864-30a53570d86f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:29.260492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647859e-f318-437a-9864-30a53570d86f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:29.271974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647859e-f318-437a-9864-30a53570d86f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-b647859e-f318-437a-9864-30a53570d86f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:29.280586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:29.281538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:29.282442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:35:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:29.287362 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647859e-f318-437a-9864-30a53570d86f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:29.345582 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1124ff9-be4d-4147-bdbb-f7cc2c94c7b5 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:29.347843 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:29.367834 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:29.383601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169893e-677f-4fb3-8a53-906d49a7d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:29.384463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169893e-677f-4fb3-8a53-906d49a7d05f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:29.384740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169893e-677f-4fb3-8a53-906d49a7d05f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:29.385004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169893e-677f-4fb3-8a53-906d49a7d05f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:29.386735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647859e-f318-437a-9864-30a53570d86f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:29.419431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b777-af3b-4bdc-a881-f15f6554e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:29.420050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b777-af3b-4bdc-a881-f15f6554e881 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:29.420351 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b777-af3b-4bdc-a881-f15f6554e881 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:29.420644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b777-af3b-4bdc-a881-f15f6554e881 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:29.421303 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b777-af3b-4bdc-a881-f15f6554e881 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:29.421603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b777-af3b-4bdc-a881-f15f6554e881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:29.423005 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b777-af3b-4bdc-a881-f15f6554e881 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:29.435103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:29.441751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b777-af3b-4bdc-a881-f15f6554e881 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:29.456803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169893e-677f-4fb3-8a53-906d49a7d05f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:29.458359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169893e-677f-4fb3-8a53-906d49a7d05f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:29.539269 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b777-af3b-4bdc-a881-f15f6554e881 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:29.552538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:29.558433 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169893e-677f-4fb3-8a53-906d49a7d05f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:29.561554 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:29.573640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa968-a5f0-4284-be8e-5f794d727cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:29.574278 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa968-a5f0-4284-be8e-5f794d727cdb None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:29.574554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa968-a5f0-4284-be8e-5f794d727cdb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:29.574825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa968-a5f0-4284-be8e-5f794d727cdb None admin] PATH_INFO: `/v3/users/506323730e1e4e22bdf37b533b4f458b` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:29.575997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa968-a5f0-4284-be8e-5f794d727cdb None admin] RBAC: Authorizing `identity:get_user(user_id=506323730e1e4e22bdf37b533b4f458b)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:29.577470 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa968-a5f0-4284-be8e-5f794d727cdb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:29.579388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:29 2022] GET /identity/v3/users/506323730e1e4e22bdf37b533b4f458b => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:29.585160 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa968-a5f0-4284-be8e-5f794d727cdb None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:29.599176 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:29.658735 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169893e-677f-4fb3-8a53-906d49a7d05f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:29.688396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa968-a5f0-4284-be8e-5f794d727cdb None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:29.692705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72a418-ebf9-499d-a5c7-64b398f3ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:29.693557 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72a418-ebf9-499d-a5c7-64b398f3ae3e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:29.693889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72a418-ebf9-499d-a5c7-64b398f3ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:29.694154 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72a418-ebf9-499d-a5c7-64b398f3ae3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:29.712977 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72a418-ebf9-499d-a5c7-64b398f3ae3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:29.714463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72a418-ebf9-499d-a5c7-64b398f3ae3e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:29.722432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1262d8b-bf5c-44d2-9cb0-b8aaa643d216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:29.723101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1262d8b-bf5c-44d2-9cb0-b8aaa643d216 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:29.723401 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1262d8b-bf5c-44d2-9cb0-b8aaa643d216 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:29.723668 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1262d8b-bf5c-44d2-9cb0-b8aaa643d216 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:29.724811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1262d8b-bf5c-44d2-9cb0-b8aaa643d216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:29.726348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1262d8b-bf5c-44d2-9cb0-b8aaa643d216 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:29.727971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-1a72a418-ebf9-499d-a5c7-64b398f3ae3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:35:29.732822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:29.733663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:35:29.734467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:35:29.734467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:35:29.734467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:35:29.734651 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1262d8b-bf5c-44d2-9cb0-b8aaa643d216 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:29.734717 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:35:29.739426 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72a418-ebf9-499d-a5c7-64b398f3ae3e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:29.794731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:29.813486 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:29.833335 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1262d8b-bf5c-44d2-9cb0-b8aaa643d216 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:29.839500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72a418-ebf9-499d-a5c7-64b398f3ae3e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:29.866348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442a9d9-f11e-480c-88a6-e1780a9df06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:29.866962 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442a9d9-f11e-480c-88a6-e1780a9df06e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:29.867293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442a9d9-f11e-480c-88a6-e1780a9df06e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:29.867560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442a9d9-f11e-480c-88a6-e1780a9df06e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:29.868313 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442a9d9-f11e-480c-88a6-e1780a9df06e None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.domain.id=default, user.id=506323730e1e4e22bdf37b533b4f458b) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:29.868616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442a9d9-f11e-480c-88a6-e1780a9df06e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:29.870074 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442a9d9-f11e-480c-88a6-e1780a9df06e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:29.872659 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b974a7e-b773-4354-bbd8-42a050b17949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:29.873258 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b974a7e-b773-4354-bbd8-42a050b17949 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:29.873534 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b974a7e-b773-4354-bbd8-42a050b17949 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:29.873804 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b974a7e-b773-4354-bbd8-42a050b17949 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:29.874439 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b974a7e-b773-4354-bbd8-42a050b17949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:29.874740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b974a7e-b773-4354-bbd8-42a050b17949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:29.876155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b974a7e-b773-4354-bbd8-42a050b17949 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:29.876806 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Tue Aug 30 11:35:29 2022] GET /identity/v3/role_assignments?user.id=506323730e1e4e22bdf37b533b4f458b&scope.domain.id=default&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:29.896344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:32.324679 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d5d39-128e-4e84-8832-e4146db61bf9 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.325574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d5d39-128e-4e84-8832-e4146db61bf9 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.325997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d5d39-128e-4e84-8832-e4146db61bf9 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.327531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:32.333286 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dcc12-4596-47ed-92e7-b92322a0d380 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.334077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dcc12-4596-47ed-92e7-b92322a0d380 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.334550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dcc12-4596-47ed-92e7-b92322a0d380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.364135 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-da8dcc12-4596-47ed-92e7-b92322a0d380 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:32.365664 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-da8dcc12-4596-47ed-92e7-b92322a0d380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:32.460376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:35:32.481762 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ec435-9c84-463f-b5d7-3ec3a7a39576 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.482051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ec435-9c84-463f-b5d7-3ec3a7a39576 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.482327 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ec435-9c84-463f-b5d7-3ec3a7a39576 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.483998 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 13/24] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:32.489096 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dd544-d41c-4129-a989-3e6a9d94aa87 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.489382 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dd544-d41c-4129-a989-3e6a9d94aa87 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.489658 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dd544-d41c-4129-a989-3e6a9d94aa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.513698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-879dd544-d41c-4129-a989-3e6a9d94aa87 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:32.514999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-879dd544-d41c-4129-a989-3e6a9d94aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:32.574132 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:32.577492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550fd03-2e0f-480a-8c17-15483f211948 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.577716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550fd03-2e0f-480a-8c17-15483f211948 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.578053 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550fd03-2e0f-480a-8c17-15483f211948 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.579815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 14/25] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:35:32.585619 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df93ec-635e-4c20-b31d-c6233192f511 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.585956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df93ec-635e-4c20-b31d-c6233192f511 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.586224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df93ec-635e-4c20-b31d-c6233192f511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.618676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-92df93ec-635e-4c20-b31d-c6233192f511 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:32.619900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-92df93ec-635e-4c20-b31d-c6233192f511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:32.642218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 12/26] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:32.729704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 15/27] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:32.756718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effbfce-21b9-4635-9a2e-a1f3cfb0d304 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.757001 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effbfce-21b9-4635-9a2e-a1f3cfb0d304 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.757461 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effbfce-21b9-4635-9a2e-a1f3cfb0d304 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.759233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 13/28] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:32.764889 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56618e32-48a0-4dc4-97e1-202d1d6abd80 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.765178 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56618e32-48a0-4dc4-97e1-202d1d6abd80 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.765427 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56618e32-48a0-4dc4-97e1-202d1d6abd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.786495 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-56618e32-48a0-4dc4-97e1-202d1d6abd80 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:32.787808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-56618e32-48a0-4dc4-97e1-202d1d6abd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:32.869707 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 16/29] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:32.935982 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af329942-f504-44d9-9188-d8c8a553fea0 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.936608 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af329942-f504-44d9-9188-d8c8a553fea0 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.937075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af329942-f504-44d9-9188-d8c8a553fea0 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.938779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 14/30] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:32.944687 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbc6f0-16a2-4a41-84cc-5a12c95a29d8 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:32.945011 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbc6f0-16a2-4a41-84cc-5a12c95a29d8 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:32.945286 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbc6f0-16a2-4a41-84cc-5a12c95a29d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:32.952824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:32.970413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-58bbc6f0-16a2-4a41-84cc-5a12c95a29d8 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:32.971605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-58bbc6f0-16a2-4a41-84cc-5a12c95a29d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:33.050432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 17/31] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:33.060312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af329942-f504-44d9-9188-d8c8a553fea0 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:33.176998 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbc6f0-16a2-4a41-84cc-5a12c95a29d8 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:33.220829 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af329942-f504-44d9-9188-d8c8a553fea0 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:33.268235 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d743-a252-4528-b5e0-739dc2c8a794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:33.268852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d743-a252-4528-b5e0-739dc2c8a794 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:33.269126 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d743-a252-4528-b5e0-739dc2c8a794 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:33.269392 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d743-a252-4528-b5e0-739dc2c8a794 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:33.270126 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834d743-a252-4528-b5e0-739dc2c8a794 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:33.272186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834d743-a252-4528-b5e0-739dc2c8a794 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:33.303290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbc6f0-16a2-4a41-84cc-5a12c95a29d8 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:33.343451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df31766-cf64-4301-b209-4bb3a7987214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:33.344223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df31766-cf64-4301-b209-4bb3a7987214 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:33.344398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df31766-cf64-4301-b209-4bb3a7987214 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:33.344641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df31766-cf64-4301-b209-4bb3a7987214 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:33.346591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:33.358308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df31766-cf64-4301-b209-4bb3a7987214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-4df31766-cf64-4301-b209-4bb3a7987214 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:33.363708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:33.364534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:33.365275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 18/32] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:35:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:33.366009 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df31766-cf64-4301-b209-4bb3a7987214 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:33.415287 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 15/33] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Aug 30 11:35:33 2022] POST /identity/v3/roles => generated 219 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:35:33.416217 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d743-a252-4528-b5e0-739dc2c8a794 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:33.461109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:33.518835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df31766-cf64-4301-b209-4bb3a7987214 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:33.541745 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d743-a252-4528-b5e0-739dc2c8a794 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:33.566885 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12c953-faf4-4b53-9c24-55678dd87f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:33.567570 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12c953-faf4-4b53-9c24-55678dd87f86 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:33.567839 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12c953-faf4-4b53-9c24-55678dd87f86 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:33.568106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12c953-faf4-4b53-9c24-55678dd87f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:33.579897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12c953-faf4-4b53-9c24-55678dd87f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:33.581529 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12c953-faf4-4b53-9c24-55678dd87f86 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:33.585839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1cdd-a896-4f33-945b-39647a5aef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:33.586690 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1cdd-a896-4f33-945b-39647a5aef54 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:33.587193 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1cdd-a896-4f33-945b-39647a5aef54 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:33.587632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1cdd-a896-4f33-945b-39647a5aef54 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:33.588514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a1cdd-a896-4f33-945b-39647a5aef54 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:33.590569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a1cdd-a896-4f33-945b-39647a5aef54 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-3e12c953-faf4-4b53-9c24-55678dd87f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:35:33.599047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:33.600150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:35:33.600965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:35:33.600965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:35:33.600965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:33.600965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 19/34] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:33.601609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12c953-faf4-4b53-9c24-55678dd87f86 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:33.610382 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 16/35] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Aug 30 11:35:33 2022] POST /identity/v3/roles => generated 225 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:35:33.611647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1cdd-a896-4f33-945b-39647a5aef54 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:33.657578 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:33.724872 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12c953-faf4-4b53-9c24-55678dd87f86 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:33.733596 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:33.743106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1cdd-a896-4f33-945b-39647a5aef54 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:33.765670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b319d-e152-4890-b86c-87739507e86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:33.766209 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b319d-e152-4890-b86c-87739507e86c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:33.766493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b319d-e152-4890-b86c-87739507e86c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:33.766762 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b319d-e152-4890-b86c-87739507e86c None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:33.767488 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b319d-e152-4890-b86c-87739507e86c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:33.767792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b319d-e152-4890-b86c-87739507e86c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:33.769346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b319d-e152-4890-b86c-87739507e86c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:33.779375 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 20/36] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:33.780196 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b319d-e152-4890-b86c-87739507e86c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:33.784351 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f210432-78c6-4e4b-8ad8-771d633d08c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:33.785275 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f210432-78c6-4e4b-8ad8-771d633d08c9 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:33.785763 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f210432-78c6-4e4b-8ad8-771d633d08c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:33.786216 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f210432-78c6-4e4b-8ad8-771d633d08c9 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:33.787136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f210432-78c6-4e4b-8ad8-771d633d08c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:33.788798 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f210432-78c6-4e4b-8ad8-771d633d08c9 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:33.808752 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 17/37] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Aug 30 11:35:33 2022] POST /identity/v3/roles => generated 223 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:35:33.810087 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f210432-78c6-4e4b-8ad8-771d633d08c9 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:33.916964 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b319d-e152-4890-b86c-87739507e86c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:33.929986 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:33.955474 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f210432-78c6-4e4b-8ad8-771d633d08c9 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:33.964051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405eb9c-8001-45f6-b751-84a7408009db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:33.964728 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405eb9c-8001-45f6-b751-84a7408009db None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:33.964949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405eb9c-8001-45f6-b751-84a7408009db None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:33.965215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405eb9c-8001-45f6-b751-84a7408009db None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:33.965851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405eb9c-8001-45f6-b751-84a7408009db None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:33.966151 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405eb9c-8001-45f6-b751-84a7408009db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:33.967598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405eb9c-8001-45f6-b751-84a7408009db None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:33.985465 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 21/38] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:33.986641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c0d3b-7929-427b-8dc8-385c8cac663e None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:33.986918 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c0d3b-7929-427b-8dc8-385c8cac663e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:33.987218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c0d3b-7929-427b-8dc8-385c8cac663e None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:33.988667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 22/39] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:33.994049 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c0d3b-7929-427b-8dc8-385c8cac663e None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:34.000744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf57665-ebf7-402b-8912-6bbf725e6db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:34.001405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf57665-ebf7-402b-8912-6bbf725e6db0 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:34.001692 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf57665-ebf7-402b-8912-6bbf725e6db0 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:34.002000 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf57665-ebf7-402b-8912-6bbf725e6db0 None admin] PATH_INFO: `/v3/users/506323730e1e4e22bdf37b533b4f458b` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:34.003601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf57665-ebf7-402b-8912-6bbf725e6db0 None admin] RBAC: Authorizing `identity:get_user(user_id=506323730e1e4e22bdf37b533b4f458b)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:34.005098 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf57665-ebf7-402b-8912-6bbf725e6db0 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:34.006796 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 18/40] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:33 2022] GET /identity/v3/users/506323730e1e4e22bdf37b533b4f458b => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:34.008300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46edbe6-36f4-48e4-abab-2aa2d9d34417 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:34.008607 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46edbe6-36f4-48e4-abab-2aa2d9d34417 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:34.008860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46edbe6-36f4-48e4-abab-2aa2d9d34417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:34.030503 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-c46edbe6-36f4-48e4-abab-2aa2d9d34417 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:34.032346 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c46edbe6-36f4-48e4-abab-2aa2d9d34417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:34.050580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:34.100733 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 19/41] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:34.101654 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46edbe6-36f4-48e4-abab-2aa2d9d34417 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:34.114373 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c0d3b-7929-427b-8dc8-385c8cac663e None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:34.155026 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dd407-9d0a-4389-9410-377adaf25e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:34.155685 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dd407-9d0a-4389-9410-377adaf25e49 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:34.155965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dd407-9d0a-4389-9410-377adaf25e49 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:34.156238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dd407-9d0a-4389-9410-377adaf25e49 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:34.156893 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2dd407-9d0a-4389-9410-377adaf25e49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:34.158366 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2dd407-9d0a-4389-9410-377adaf25e49 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:34.207716 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46edbe6-36f4-48e4-abab-2aa2d9d34417 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:34.242156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d94c9-751d-4a1f-b91a-00f48183daf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:34.242748 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d94c9-751d-4a1f-b91a-00f48183daf7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:34.243028 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d94c9-751d-4a1f-b91a-00f48183daf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:34.243335 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d94c9-751d-4a1f-b91a-00f48183daf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:34.244473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d94c9-751d-4a1f-b91a-00f48183daf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:34.245951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d94c9-751d-4a1f-b91a-00f48183daf7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:34.247761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 20/42] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:34.253147 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d94c9-751d-4a1f-b91a-00f48183daf7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:34.277245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:34.307013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 23/43] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:35:33 2022] POST /identity/v3/projects => generated 301 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:35:34.362276 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d94c9-751d-4a1f-b91a-00f48183daf7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:34.366901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dd407-9d0a-4389-9410-377adaf25e49 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:34.392603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:34.402453 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce47e95-015e-42cc-84ea-2690cc782538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:34.403080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce47e95-015e-42cc-84ea-2690cc782538 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:34.403410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce47e95-015e-42cc-84ea-2690cc782538 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:34.403691 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce47e95-015e-42cc-84ea-2690cc782538 None admin] PATH_INFO: `/v3/domains/default/users/506323730e1e4e22bdf37b533b4f458b/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:34.405843 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce47e95-015e-42cc-84ea-2690cc782538 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=506323730e1e4e22bdf37b533b4f458b, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:34.407238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce47e95-015e-42cc-84ea-2690cc782538 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:34.428642 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 21/44] 10.222.0.16 () {58 vars in 1361 bytes} [Tue Aug 30 11:35:34 2022] PUT /identity/v3/domains/default/users/506323730e1e4e22bdf37b533b4f458b/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 11:35:34.602821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dd407-9d0a-4389-9410-377adaf25e49 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:34.639921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3458cb-f495-4704-86e4-7c825cbb44e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:34.640562 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3458cb-f495-4704-86e4-7c825cbb44e5 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:34.640835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3458cb-f495-4704-86e4-7c825cbb44e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:34.641107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3458cb-f495-4704-86e4-7c825cbb44e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:34.642361 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3458cb-f495-4704-86e4-7c825cbb44e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:34.643971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3458cb-f495-4704-86e4-7c825cbb44e5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:34.645553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 24/45] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:34.651561 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce47e95-015e-42cc-84ea-2690cc782538 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:34.738951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:34.758550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce47e95-015e-42cc-84ea-2690cc782538 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:34.793984 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3580a-4079-48a9-bfe0-d4915ac9c62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:34.794601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3580a-4079-48a9-bfe0-d4915ac9c62d None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:34.794877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3580a-4079-48a9-bfe0-d4915ac9c62d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:34.795157 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3580a-4079-48a9-bfe0-d4915ac9c62d None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:34.795829 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3580a-4079-48a9-bfe0-d4915ac9c62d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:34.797291 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3580a-4079-48a9-bfe0-d4915ac9c62d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:34.825644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 22/46] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:35:34 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:35:36.295927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeca86-bf4b-4253-882c-795afb52872c None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:36.296471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeca86-bf4b-4253-882c-795afb52872c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:36.296471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeca86-bf4b-4253-882c-795afb52872c None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:36.297827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 25/47] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:36.303364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b2aa9-8a91-43a9-95ee-4f7dfb02370b None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:36.303788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b2aa9-8a91-43a9-95ee-4f7dfb02370b None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:36.303876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b2aa9-8a91-43a9-95ee-4f7dfb02370b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:36.322749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-ae5b2aa9-8a91-43a9-95ee-4f7dfb02370b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:36.323956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-ae5b2aa9-8a91-43a9-95ee-4f7dfb02370b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:36.392268 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 23/48] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:36.659219 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeca86-bf4b-4253-882c-795afb52872c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:36.707507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d158cc-f088-4ced-be7f-4e70e706beea None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:36.707915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d158cc-f088-4ced-be7f-4e70e706beea None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:36.708188 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d158cc-f088-4ced-be7f-4e70e706beea None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:36.709548 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 24/49] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:35:36.715229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462ad27-c3d4-44e5-9d19-ba420c77739a None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:36.715539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462ad27-c3d4-44e5-9d19-ba420c77739a None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:36.715806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462ad27-c3d4-44e5-9d19-ba420c77739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:36.734131 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-b462ad27-c3d4-44e5-9d19-ba420c77739a None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:36.735346 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-b462ad27-c3d4-44e5-9d19-ba420c77739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:36.750585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:36.764062 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeca86-bf4b-4253-882c-795afb52872c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:36.793843 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:36.797927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94c9d7-c47f-4209-b6ca-0ae42356597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:36.798574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94c9d7-c47f-4209-b6ca-0ae42356597d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:36.798891 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94c9d7-c47f-4209-b6ca-0ae42356597d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:36.799160 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94c9d7-c47f-4209-b6ca-0ae42356597d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:36.803811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 25/50] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:35:36.810329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94c9d7-c47f-4209-b6ca-0ae42356597d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-2a94c9d7-c47f-4209-b6ca-0ae42356597d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:36.815450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:36.816459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:36.817348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 26/51] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:35:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:36.822276 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462ad27-c3d4-44e5-9d19-ba420c77739a None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:36.920887 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462ad27-c3d4-44e5-9d19-ba420c77739a None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:36.956796 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c2bf5-17c4-4956-a056-be4c07133ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:36.957406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c2bf5-17c4-4956-a056-be4c07133ce5 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:36.957688 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c2bf5-17c4-4956-a056-be4c07133ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:36.957961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c2bf5-17c4-4956-a056-be4c07133ce5 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:36.958637 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c2bf5-17c4-4956-a056-be4c07133ce5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:36.958947 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c2bf5-17c4-4956-a056-be4c07133ce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:36.960442 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c2bf5-17c4-4956-a056-be4c07133ce5 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:36.972099 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 26/52] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:36.977802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94c9d7-c47f-4209-b6ca-0ae42356597d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:37.065371 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c2bf5-17c4-4956-a056-be4c07133ce5 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:37.079944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94c9d7-c47f-4209-b6ca-0ae42356597d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:37.112418 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082bb65-b52c-40c8-8334-88dc1648532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:37.113065 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082bb65-b52c-40c8-8334-88dc1648532a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:37.113341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082bb65-b52c-40c8-8334-88dc1648532a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:37.113612 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082bb65-b52c-40c8-8334-88dc1648532a None admin] PATH_INFO: `/v3/users/506323730e1e4e22bdf37b533b4f458b` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:37.114887 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082bb65-b52c-40c8-8334-88dc1648532a None admin] RBAC: Authorizing `identity:get_user(user_id=506323730e1e4e22bdf37b533b4f458b)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:37.116422 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082bb65-b52c-40c8-8334-88dc1648532a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:37.118049 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 27/53] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:36 2022] GET /identity/v3/users/506323730e1e4e22bdf37b533b4f458b => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:37.122007 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:37.124278 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082bb65-b52c-40c8-8334-88dc1648532a None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:37.142684 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:37.165827 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c2bf5-17c4-4956-a056-be4c07133ce5 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:37.228704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082bb65-b52c-40c8-8334-88dc1648532a None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:37.266174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73634b-c859-4d8f-b75f-9590e035d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:37.266935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73634b-c859-4d8f-b75f-9590e035d0bb None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:37.267239 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73634b-c859-4d8f-b75f-9590e035d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:37.267511 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73634b-c859-4d8f-b75f-9590e035d0bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:37.268747 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73634b-c859-4d8f-b75f-9590e035d0bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:37.270245 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73634b-c859-4d8f-b75f-9590e035d0bb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:37.271893 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 28/54] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:37.277805 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73634b-c859-4d8f-b75f-9590e035d0bb None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:37.309555 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376247ae-5f99-4c41-9abf-2dadd260c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:37.310201 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376247ae-5f99-4c41-9abf-2dadd260c647 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:37.310487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376247ae-5f99-4c41-9abf-2dadd260c647 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:37.310760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376247ae-5f99-4c41-9abf-2dadd260c647 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:37.312160 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376247ae-5f99-4c41-9abf-2dadd260c647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:37.313821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376247ae-5f99-4c41-9abf-2dadd260c647 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:37.315531 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:37.321108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376247ae-5f99-4c41-9abf-2dadd260c647 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:37.367429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:37.383724 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73634b-c859-4d8f-b75f-9590e035d0bb None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:37.418130 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e09a0a-d90e-4512-bab1-e0bec16661ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:37.418744 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e09a0a-d90e-4512-bab1-e0bec16661ec None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:37.419012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e09a0a-d90e-4512-bab1-e0bec16661ec None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:37.419342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e09a0a-d90e-4512-bab1-e0bec16661ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:37.420064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e09a0a-d90e-4512-bab1-e0bec16661ec None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.domain.id=default, user.id=506323730e1e4e22bdf37b533b4f458b) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:37.420367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e09a0a-d90e-4512-bab1-e0bec16661ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:37.421837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e09a0a-d90e-4512-bab1-e0bec16661ec None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:37.422686 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376247ae-5f99-4c41-9abf-2dadd260c647 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:37.427743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 29/56] 10.222.0.16 () {56 vars in 1322 bytes} [Tue Aug 30 11:35:37 2022] GET /identity/v3/role_assignments?user.id=506323730e1e4e22bdf37b533b4f458b&scope.domain.id=default&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:37.448240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:37.460450 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf1c23-7a79-4a86-b6d9-ce8b3a5ca76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:37.461071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf1c23-7a79-4a86-b6d9-ce8b3a5ca76a None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:37.461348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf1c23-7a79-4a86-b6d9-ce8b3a5ca76a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:37.461616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf1c23-7a79-4a86-b6d9-ce8b3a5ca76a None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:37.462266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf1c23-7a79-4a86-b6d9-ce8b3a5ca76a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:37.463780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf1c23-7a79-4a86-b6d9-ce8b3a5ca76a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:37.493943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 28/57] 10.222.0.16 () {60 vars in 1140 bytes} [Tue Aug 30 11:35:37 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:35:39.375970 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a8c19-1ab0-46a6-8cbd-f26afdceb867 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:39.376514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a8c19-1ab0-46a6-8cbd-f26afdceb867 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:39.376514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a8c19-1ab0-46a6-8cbd-f26afdceb867 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:39.377882 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 30/58] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:39.384351 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e663a7a-5917-462d-837c-32a566e0ca10 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:39.385079 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e663a7a-5917-462d-837c-32a566e0ca10 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:39.385380 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e663a7a-5917-462d-837c-32a566e0ca10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:39.413831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-8e663a7a-5917-462d-837c-32a566e0ca10 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:39.415017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-8e663a7a-5917-462d-837c-32a566e0ca10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:39.485746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 29/59] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:39.750334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a8c19-1ab0-46a6-8cbd-f26afdceb867 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:39.859575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a8c19-1ab0-46a6-8cbd-f26afdceb867 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:39.901869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f079711-cf1a-4dd4-a329-adc7a1da1004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:39.902500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f079711-cf1a-4dd4-a329-adc7a1da1004 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:39.902770 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f079711-cf1a-4dd4-a329-adc7a1da1004 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:39.903033 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f079711-cf1a-4dd4-a329-adc7a1da1004 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:39.904334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f079711-cf1a-4dd4-a329-adc7a1da1004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:39.905816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f079711-cf1a-4dd4-a329-adc7a1da1004 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:39.907400 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 31/60] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:39.913250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e663a7a-5917-462d-837c-32a566e0ca10 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:40.018682 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e663a7a-5917-462d-837c-32a566e0ca10 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:40.026541 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:40.059305 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbf4ea-3aa0-4763-a2ca-3f06cb168ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:40.059920 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbf4ea-3aa0-4763-a2ca-3f06cb168ce2 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:40.060203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbf4ea-3aa0-4763-a2ca-3f06cb168ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:40.060475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbf4ea-3aa0-4763-a2ca-3f06cb168ce2 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:40.061135 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbf4ea-3aa0-4763-a2ca-3f06cb168ce2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:40.062595 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbf4ea-3aa0-4763-a2ca-3f06cb168ce2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:40.092324 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 30/61] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:35:39 2022] POST /identity/v3/users => generated 275 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:35:43.126341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243c00a-7726-4c7b-8aef-19160a4d06a1 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.126341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243c00a-7726-4c7b-8aef-19160a4d06a1 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.126341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243c00a-7726-4c7b-8aef-19160a4d06a1 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:43.127472 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 32/62] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:43.133170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8827280-4625-409a-8f56-ebeec3527a21 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.134033 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8827280-4625-409a-8f56-ebeec3527a21 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.134524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8827280-4625-409a-8f56-ebeec3527a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:43.164961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-a8827280-4625-409a-8f56-ebeec3527a21 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:43.166501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-a8827280-4625-409a-8f56-ebeec3527a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:43.261425 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 31/63] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:43.287017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286654-82a4-4fb5-b8ad-5b8230151f06 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.287691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286654-82a4-4fb5-b8ad-5b8230151f06 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.288166 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286654-82a4-4fb5-b8ad-5b8230151f06 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:43.289722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 33/64] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:43.294872 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5410495-c764-40e9-9447-cbac2421110c None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.295399 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5410495-c764-40e9-9447-cbac2421110c None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.295871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5410495-c764-40e9-9447-cbac2421110c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:43.320173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-b5410495-c764-40e9-9447-cbac2421110c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:43.321725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-b5410495-c764-40e9-9447-cbac2421110c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:43.404947 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 32/65] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:43.503828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d2e67-3704-46f6-a5cd-d5012cd96535 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.504454 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d2e67-3704-46f6-a5cd-d5012cd96535 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.504916 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d2e67-3704-46f6-a5cd-d5012cd96535 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:43.506524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 34/66] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:43.511898 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a39390-5dec-4978-801e-dec23ec9a2be None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.512403 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a39390-5dec-4978-801e-dec23ec9a2be None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.512867 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a39390-5dec-4978-801e-dec23ec9a2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:43.535079 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-f2a39390-5dec-4978-801e-dec23ec9a2be None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:43.536614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-f2a39390-5dec-4978-801e-dec23ec9a2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:43.560591 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:43.624369 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 33/67] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:43.672439 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add47064-4c14-4f6c-9d66-6da9d05c14e7 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.672722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add47064-4c14-4f6c-9d66-6da9d05c14e7 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.672906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add47064-4c14-4f6c-9d66-6da9d05c14e7 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:43.674283 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 35/68] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:43.676218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a39390-5dec-4978-801e-dec23ec9a2be None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:43.680550 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcc113-03aa-46fd-92e7-bda45dd6836b None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.681078 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcc113-03aa-46fd-92e7-bda45dd6836b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.681537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcc113-03aa-46fd-92e7-bda45dd6836b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:43.708544 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-14dcc113-03aa-46fd-92e7-bda45dd6836b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:43.710140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-14dcc113-03aa-46fd-92e7-bda45dd6836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:43.732538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:43.813361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a39390-5dec-4978-801e-dec23ec9a2be None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:43.816039 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 36/69] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:35:43.817417 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcc113-03aa-46fd-92e7-bda45dd6836b None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:43.849101 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04026f7d-c346-49a0-bbed-dc78bc035a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:43.850034 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04026f7d-c346-49a0-bbed-dc78bc035a36 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.850453 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04026f7d-c346-49a0-bbed-dc78bc035a36 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.850864 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04026f7d-c346-49a0-bbed-dc78bc035a36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:43.862602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04026f7d-c346-49a0-bbed-dc78bc035a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-04026f7d-c346-49a0-bbed-dc78bc035a36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:43.869396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:43.870245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:43.871012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:35:43.873322 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 34/70] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:43.881605 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04026f7d-c346-49a0-bbed-dc78bc035a36 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:43.910453 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:43.964112 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcc113-03aa-46fd-92e7-bda45dd6836b None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:43.997739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a3651-d9a4-4a5d-ad91-5d7ab4aeb72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:43.998638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a3651-d9a4-4a5d-ad91-5d7ab4aeb72a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:43.999045 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a3651-d9a4-4a5d-ad91-5d7ab4aeb72a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:43.999471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a3651-d9a4-4a5d-ad91-5d7ab4aeb72a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.010301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a3651-d9a4-4a5d-ad91-5d7ab4aeb72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-521a3651-d9a4-4a5d-ad91-5d7ab4aeb72a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:44.015133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:44.016191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:44.017253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:44.020868 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 37/71] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:35:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:44.022132 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adada1c2-4503-43e9-9ced-0af48da2b064 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.022446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adada1c2-4503-43e9-9ced-0af48da2b064 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.022706 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adada1c2-4503-43e9-9ced-0af48da2b064 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.024484 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 38/72] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:44.025306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adada1c2-4503-43e9-9ced-0af48da2b064 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.025693 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04026f7d-c346-49a0-bbed-dc78bc035a36 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.037511 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:44.063675 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0522e53-0ded-42fe-b001-ceb50a30a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.064289 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0522e53-0ded-42fe-b001-ceb50a30a322 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.064568 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0522e53-0ded-42fe-b001-ceb50a30a322 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.064839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0522e53-0ded-42fe-b001-ceb50a30a322 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.065508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0522e53-0ded-42fe-b001-ceb50a30a322 None admin] RBAC: Adding query filter params (name=member) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:44.065814 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0522e53-0ded-42fe-b001-ceb50a30a322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:44.067237 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0522e53-0ded-42fe-b001-ceb50a30a322 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:44.075908 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 35/73] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:35:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:44.081474 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0522e53-0ded-42fe-b001-ceb50a30a322 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.142962 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adada1c2-4503-43e9-9ced-0af48da2b064 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.160921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:44.177076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba5f15-581a-4e3f-b3f3-41e796a788d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.177720 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba5f15-581a-4e3f-b3f3-41e796a788d0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.178027 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba5f15-581a-4e3f-b3f3-41e796a788d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.178298 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba5f15-581a-4e3f-b3f3-41e796a788d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.187091 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0522e53-0ded-42fe-b001-ceb50a30a322 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.188283 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba5f15-581a-4e3f-b3f3-41e796a788d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-4dba5f15-581a-4e3f-b3f3-41e796a788d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:44.193159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:44.194088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:44.194849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 39/74] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:44.195926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32942ea3-dd25-4757-98a1-5cfa85b7c976 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.195926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32942ea3-dd25-4757-98a1-5cfa85b7c976 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.196052 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32942ea3-dd25-4757-98a1-5cfa85b7c976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.214315 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-32942ea3-dd25-4757-98a1-5cfa85b7c976 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:44.215457 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-32942ea3-dd25-4757-98a1-5cfa85b7c976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:44.224509 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dd07b-55d7-40df-b79d-3cd3c224b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.225193 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dd07b-55d7-40df-b79d-3cd3c224b68e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.225468 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dd07b-55d7-40df-b79d-3cd3c224b68e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.225744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dd07b-55d7-40df-b79d-3cd3c224b68e None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.226420 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dd07b-55d7-40df-b79d-3cd3c224b68e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:44.226711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dd07b-55d7-40df-b79d-3cd3c224b68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:44.228140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dd07b-55d7-40df-b79d-3cd3c224b68e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:44.236188 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 36/75] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:44.236966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dd07b-55d7-40df-b79d-3cd3c224b68e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.274311 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:44.281842 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 40/76] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:44.282650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32942ea3-dd25-4757-98a1-5cfa85b7c976 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.343176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dd07b-55d7-40df-b79d-3cd3c224b68e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.377937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd393a4-8d53-4b7b-b543-15aaf0868036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.378495 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32942ea3-dd25-4757-98a1-5cfa85b7c976 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.378599 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd393a4-8d53-4b7b-b543-15aaf0868036 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.378835 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd393a4-8d53-4b7b-b543-15aaf0868036 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.379108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd393a4-8d53-4b7b-b543-15aaf0868036 None admin] PATH_INFO: `/v3/users/3092900ab6da4b199709df63140b22ea` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.382210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:44.397442 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd393a4-8d53-4b7b-b543-15aaf0868036 None admin] RBAC: Authorizing `identity:get_user(user_id=3092900ab6da4b199709df63140b22ea)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:44.398908 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd393a4-8d53-4b7b-b543-15aaf0868036 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:44.400768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 37/77] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/users/3092900ab6da4b199709df63140b22ea => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:44.406057 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd393a4-8d53-4b7b-b543-15aaf0868036 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.418084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cb5c2-c35d-4c83-a772-89f1f3b07470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.419174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cb5c2-c35d-4c83-a772-89f1f3b07470 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.419465 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cb5c2-c35d-4c83-a772-89f1f3b07470 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.419732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cb5c2-c35d-4c83-a772-89f1f3b07470 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.431883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cb5c2-c35d-4c83-a772-89f1f3b07470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-478cb5c2-c35d-4c83-a772-89f1f3b07470 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:44.436862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:44.437716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:44.438459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 41/78] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:44.439096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cb5c2-c35d-4c83-a772-89f1f3b07470 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.496189 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:44.524741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd393a4-8d53-4b7b-b543-15aaf0868036 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.541768 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cb5c2-c35d-4c83-a772-89f1f3b07470 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.558809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f26441-e930-4164-8dea-d346f8c02955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.559444 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f26441-e930-4164-8dea-d346f8c02955 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.559725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f26441-e930-4164-8dea-d346f8c02955 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.559990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f26441-e930-4164-8dea-d346f8c02955 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.560644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f26441-e930-4164-8dea-d346f8c02955 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:44.560942 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f26441-e930-4164-8dea-d346f8c02955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:44.562309 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f26441-e930-4164-8dea-d346f8c02955 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:44.570147 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 38/79] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:44.570992 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f26441-e930-4164-8dea-d346f8c02955 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.575143 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b3129-83f6-48de-8b12-32453c4eae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.575801 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b3129-83f6-48de-8b12-32453c4eae2f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.576083 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b3129-83f6-48de-8b12-32453c4eae2f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.576357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b3129-83f6-48de-8b12-32453c4eae2f None admin] PATH_INFO: `/v3/users/506323730e1e4e22bdf37b533b4f458b` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.577513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b3129-83f6-48de-8b12-32453c4eae2f None admin] RBAC: Authorizing `identity:get_user(user_id=506323730e1e4e22bdf37b533b4f458b)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:44.579334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b3129-83f6-48de-8b12-32453c4eae2f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:44.580956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 42/80] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/users/506323730e1e4e22bdf37b533b4f458b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:44.581717 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b3129-83f6-48de-8b12-32453c4eae2f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.628648 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:44.671139 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f26441-e930-4164-8dea-d346f8c02955 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.679797 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b3129-83f6-48de-8b12-32453c4eae2f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.704161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:44.706913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc63b8c-fc70-45e1-8876-3b32b49d5074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.707569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63b8c-fc70-45e1-8876-3b32b49d5074 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.707854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63b8c-fc70-45e1-8876-3b32b49d5074 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.708129 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63b8c-fc70-45e1-8876-3b32b49d5074 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.714865 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde0111-550e-496e-a031-b155621ae9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.715518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0111-550e-496e-a031-b155621ae9e5 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.715797 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0111-550e-496e-a031-b155621ae9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.716072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0111-550e-496e-a031-b155621ae9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.716749 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde0111-550e-496e-a031-b155621ae9e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:44.717055 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde0111-550e-496e-a031-b155621ae9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:44.718446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde0111-550e-496e-a031-b155621ae9e5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:44.726941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 43/81] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:44.728007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde0111-550e-496e-a031-b155621ae9e5 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.775029 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc63b8c-fc70-45e1-8876-3b32b49d5074 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:44.776464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc63b8c-fc70-45e1-8876-3b32b49d5074 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:44.825165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde0111-550e-496e-a031-b155621ae9e5 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.829303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:44.860775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54edbc0-f14d-4bfa-a575-78a721562534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:44.861413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edbc0-f14d-4bfa-a575-78a721562534 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:44.861691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edbc0-f14d-4bfa-a575-78a721562534 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:44.861963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edbc0-f14d-4bfa-a575-78a721562534 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:44.863104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54edbc0-f14d-4bfa-a575-78a721562534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:44.864655 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54edbc0-f14d-4bfa-a575-78a721562534 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:44.866299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 44/82] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:44.868196 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54edbc0-f14d-4bfa-a575-78a721562534 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.874023 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 39/83] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:44.874859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc63b8c-fc70-45e1-8876-3b32b49d5074 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:44.922562 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:44.968190 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54edbc0-f14d-4bfa-a575-78a721562534 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:44.975353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc63b8c-fc70-45e1-8876-3b32b49d5074 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:45.000959 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc19c84-e4ec-48a0-af64-ac5dc70ee2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:45.001590 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc19c84-e4ec-48a0-af64-ac5dc70ee2b9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:45.001872 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc19c84-e4ec-48a0-af64-ac5dc70ee2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:45.002134 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc19c84-e4ec-48a0-af64-ac5dc70ee2b9 None admin] PATH_INFO: `/v3/users/3092900ab6da4b199709df63140b22ea` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:45.003286 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc19c84-e4ec-48a0-af64-ac5dc70ee2b9 None admin] RBAC: Authorizing `identity:get_user(user_id=3092900ab6da4b199709df63140b22ea)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:45.004748 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc19c84-e4ec-48a0-af64-ac5dc70ee2b9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:45.006338 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 45/84] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/users/3092900ab6da4b199709df63140b22ea => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:45.007088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc19c84-e4ec-48a0-af64-ac5dc70ee2b9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:45.008550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbac2f-4815-4d49-aa02-b4f3f537ce15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:45.009159 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbac2f-4815-4d49-aa02-b4f3f537ce15 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:45.009560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbac2f-4815-4d49-aa02-b4f3f537ce15 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:45.009833 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbac2f-4815-4d49-aa02-b4f3f537ce15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:45.035473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:45.080335 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbac2f-4815-4d49-aa02-b4f3f537ce15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:45.081823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbac2f-4815-4d49-aa02-b4f3f537ce15 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:45.106681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc19c84-e4ec-48a0-af64-ac5dc70ee2b9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:45.128263 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:45.140185 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ae6c7-2713-4afa-b15c-bcd47aa515db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:45.140869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ae6c7-2713-4afa-b15c-bcd47aa515db None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:45.141160 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ae6c7-2713-4afa-b15c-bcd47aa515db None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:45.141433 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ae6c7-2713-4afa-b15c-bcd47aa515db None admin] PATH_INFO: `/v3/users/3092900ab6da4b199709df63140b22ea` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:45.142583 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ae6c7-2713-4afa-b15c-bcd47aa515db None admin] RBAC: Authorizing `identity:get_user(user_id=3092900ab6da4b199709df63140b22ea)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:45.144085 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ae6c7-2713-4afa-b15c-bcd47aa515db None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:45.145680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 46/85] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:45 2022] GET /identity/v3/users/3092900ab6da4b199709df63140b22ea => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:45.146426 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ae6c7-2713-4afa-b15c-bcd47aa515db None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:45.177860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 40/86] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:45.178727 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbac2f-4815-4d49-aa02-b4f3f537ce15 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:45.234064 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:45.247459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ae6c7-2713-4afa-b15c-bcd47aa515db None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:45.277366 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbac2f-4815-4d49-aa02-b4f3f537ce15 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:45.280581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36613e17-6894-4de5-874e-383f0218a758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:45.281191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36613e17-6894-4de5-874e-383f0218a758 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:45.281465 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36613e17-6894-4de5-874e-383f0218a758 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:45.281729 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36613e17-6894-4de5-874e-383f0218a758 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:45.282379 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36613e17-6894-4de5-874e-383f0218a758 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:45.283856 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36613e17-6894-4de5-874e-383f0218a758 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:45.310670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931aeaa-5d53-4767-b44a-b6ed036a7036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:45.311331 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 47/87] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:35:45 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:35:45.311446 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931aeaa-5d53-4767-b44a-b6ed036a7036 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:45.311638 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931aeaa-5d53-4767-b44a-b6ed036a7036 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:45.311914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931aeaa-5d53-4767-b44a-b6ed036a7036 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:45.312144 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36613e17-6894-4de5-874e-383f0218a758 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:45.313095 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931aeaa-5d53-4767-b44a-b6ed036a7036 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:45.314899 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931aeaa-5d53-4767-b44a-b6ed036a7036 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:45.321627 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 41/88] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:45 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:45.321627 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931aeaa-5d53-4767-b44a-b6ed036a7036 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:45.382060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:45.434199 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931aeaa-5d53-4767-b44a-b6ed036a7036 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:45.449302 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36613e17-6894-4de5-874e-383f0218a758 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:45.468448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:45.484489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a6d44-6be8-42b3-844c-7b28a43ac5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:45.485171 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a6d44-6be8-42b3-844c-7b28a43ac5a3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:45.485451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a6d44-6be8-42b3-844c-7b28a43ac5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:45.486081 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a6d44-6be8-42b3-844c-7b28a43ac5a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:45.500661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a8168-e6f8-48ce-9601-1aaae46faf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:45.501319 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a8168-e6f8-48ce-9601-1aaae46faf6c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:45.501598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a8168-e6f8-48ce-9601-1aaae46faf6c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:45.501869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a8168-e6f8-48ce-9601-1aaae46faf6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:45.570599 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a6d44-6be8-42b3-844c-7b28a43ac5a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:45.572699 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a6d44-6be8-42b3-844c-7b28a43ac5a3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:45.585261 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a8168-e6f8-48ce-9601-1aaae46faf6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:45.586752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a8168-e6f8-48ce-9601-1aaae46faf6c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:45.642000 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:45.695168 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 42/89] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:45.696271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a6d44-6be8-42b3-844c-7b28a43ac5a3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:45.706458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 48/90] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:45.707661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a8168-e6f8-48ce-9601-1aaae46faf6c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:45.727502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:45.809649 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a6d44-6be8-42b3-844c-7b28a43ac5a3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:45.818259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a8168-e6f8-48ce-9601-1aaae46faf6c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:45.846685 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d8bf6-0acb-4610-8ddf-9f60f6281172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:45.847326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d8bf6-0acb-4610-8ddf-9f60f6281172 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:45.847636 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d8bf6-0acb-4610-8ddf-9f60f6281172 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:45.847863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d8bf6-0acb-4610-8ddf-9f60f6281172 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:45.849135 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d8bf6-0acb-4610-8ddf-9f60f6281172 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:45.851407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d8bf6-0acb-4610-8ddf-9f60f6281172 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:45.853632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 43/91] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:45 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:45.854224 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d8bf6-0acb-4610-8ddf-9f60f6281172 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:45.855953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918166a-8e59-4e48-9115-bc52febc3baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:45.856810 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918166a-8e59-4e48-9115-bc52febc3baa None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:45.857254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918166a-8e59-4e48-9115-bc52febc3baa None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:45.857708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918166a-8e59-4e48-9115-bc52febc3baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:45.858601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918166a-8e59-4e48-9115-bc52febc3baa None admin] RBAC: Adding query filter params (role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224, user.id=3092900ab6da4b199709df63140b22ea) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:45.859188 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918166a-8e59-4e48-9115-bc52febc3baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:45.860817 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918166a-8e59-4e48-9115-bc52febc3baa None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:45.867182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 49/92] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:45 2022] GET /identity/v3/role_assignments?user.id=3092900ab6da4b199709df63140b22ea&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:45.868276 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918166a-8e59-4e48-9115-bc52febc3baa None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:45.882896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:45.992523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918166a-8e59-4e48-9115-bc52febc3baa None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:46.046713 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b04aa6-79d1-4923-a4ec-198a701c0704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:46.047395 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b04aa6-79d1-4923-a4ec-198a701c0704 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:46.047670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b04aa6-79d1-4923-a4ec-198a701c0704 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:46.047941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b04aa6-79d1-4923-a4ec-198a701c0704 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:46.049464 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b04aa6-79d1-4923-a4ec-198a701c0704 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:46.051473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b04aa6-79d1-4923-a4ec-198a701c0704 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:46.053664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 50/93] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:45 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:46.054487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b04aa6-79d1-4923-a4ec-198a701c0704 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:46.072734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:46.091431 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d8bf6-0acb-4610-8ddf-9f60f6281172 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:46.168633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b04aa6-79d1-4923-a4ec-198a701c0704 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:46.192599 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:46.208057 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1f607-deac-42cd-8489-e940b6aaa7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:46.208946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1f607-deac-42cd-8489-e940b6aaa7b7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:46.209951 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1f607-deac-42cd-8489-e940b6aaa7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:46.210866 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1f607-deac-42cd-8489-e940b6aaa7b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:46.212145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1f607-deac-42cd-8489-e940b6aaa7b7 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224, user.id=506323730e1e4e22bdf37b533b4f458b) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:46.213146 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1f607-deac-42cd-8489-e940b6aaa7b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:46.213966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f255ee5-796d-43e4-a412-b911837d4ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:46.216034 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1f607-deac-42cd-8489-e940b6aaa7b7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:46.216881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f255ee5-796d-43e4-a412-b911837d4ec9 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:46.217345 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f255ee5-796d-43e4-a412-b911837d4ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:46.217813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f255ee5-796d-43e4-a412-b911837d4ec9 None admin] PATH_INFO: `/v3/projects/c5fd9e09c605438eb4b2fb75ce124259` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:46.221065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f255ee5-796d-43e4-a412-b911837d4ec9 None admin] RBAC: Authorizing `identity:get_project(project_id=c5fd9e09c605438eb4b2fb75ce124259)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:46.222695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f255ee5-796d-43e4-a412-b911837d4ec9 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:46.228903 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 51/94] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:46 2022] GET /identity/v3/role_assignments?user.id=506323730e1e4e22bdf37b533b4f458b&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:46.229522 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 44/95] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:45 2022] GET /identity/v3/projects/c5fd9e09c605438eb4b2fb75ce124259 => generated 312 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:46.230619 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f255ee5-796d-43e4-a412-b911837d4ec9 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:46.241092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1f607-deac-42cd-8489-e940b6aaa7b7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:46.410549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1f607-deac-42cd-8489-e940b6aaa7b7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:46.455190 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75fbc2-ff80-42d4-9034-f353084b2a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:46.456137 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75fbc2-ff80-42d4-9034-f353084b2a75 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:46.456602 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75fbc2-ff80-42d4-9034-f353084b2a75 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:46.457090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75fbc2-ff80-42d4-9034-f353084b2a75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:46.458006 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75fbc2-ff80-42d4-9034-f353084b2a75 None admin] RBAC: Adding query filter params (role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=c5fd9e09c605438eb4b2fb75ce124259, user.id=3092900ab6da4b199709df63140b22ea) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:46.458657 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75fbc2-ff80-42d4-9034-f353084b2a75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:46.462278 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75fbc2-ff80-42d4-9034-f353084b2a75 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:46.472728 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 52/96] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:46 2022] GET /identity/v3/role_assignments?user.id=3092900ab6da4b199709df63140b22ea&scope.project.id=c5fd9e09c605438eb4b2fb75ce124259&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:46.484213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f255ee5-796d-43e4-a412-b911837d4ec9 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:46.528813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d1d35-262a-460d-8291-c870c76021bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:46.529794 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d1d35-262a-460d-8291-c870c76021bf None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:46.530304 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d1d35-262a-460d-8291-c870c76021bf None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:46.530867 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d1d35-262a-460d-8291-c870c76021bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:46.531840 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d1d35-262a-460d-8291-c870c76021bf None admin] RBAC: Adding query filter params (role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224, user.id=3092900ab6da4b199709df63140b22ea) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:46.532359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d1d35-262a-460d-8291-c870c76021bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:46.534006 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d1d35-262a-460d-8291-c870c76021bf None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:46.542668 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 45/97] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:46 2022] GET /identity/v3/role_assignments?user.id=3092900ab6da4b199709df63140b22ea&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:47.750424 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f759e-1e56-4e48-b7f9-7f6a620a1749 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:47.750933 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f759e-1e56-4e48-b7f9-7f6a620a1749 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:47.751006 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f759e-1e56-4e48-b7f9-7f6a620a1749 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:47.752562 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 53/98] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:47.757968 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d2a4b-b708-4a27-b011-476471a093ee None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:47.758372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d2a4b-b708-4a27-b011-476471a093ee None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:47.758535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d2a4b-b708-4a27-b011-476471a093ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:47.782549 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-3d0d2a4b-b708-4a27-b011-476471a093ee None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:47.783774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-3d0d2a4b-b708-4a27-b011-476471a093ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:47.805171 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:47.853528 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930aa16-76ee-4099-97f8-eb57ab334c6b None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:47.853862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930aa16-76ee-4099-97f8-eb57ab334c6b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:47.854066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930aa16-76ee-4099-97f8-eb57ab334c6b None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:47.855578 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 54/99] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:47.861425 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae6dc3-bc16-45f1-91c5-a0e75344f67d None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:47.861698 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae6dc3-bc16-45f1-91c5-a0e75344f67d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:47.861991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae6dc3-bc16-45f1-91c5-a0e75344f67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:47.876971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:47.885924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 46/100] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:47.886903 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-75ae6dc3-bc16-45f1-91c5-a0e75344f67d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:47.888180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-75ae6dc3-bc16-45f1-91c5-a0e75344f67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:47.955313 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 55/101] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:35:48.245834 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d2a4b-b708-4a27-b011-476471a093ee None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:48.264272 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae6dc3-bc16-45f1-91c5-a0e75344f67d None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:48.421471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae6dc3-bc16-45f1-91c5-a0e75344f67d None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:48.450905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d2a4b-b708-4a27-b011-476471a093ee None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:48.458337 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338282c-4d9a-445b-a83b-66ddb514c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:48.459299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338282c-4d9a-445b-a83b-66ddb514c09e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:48.459769 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338282c-4d9a-445b-a83b-66ddb514c09e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:48.460250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338282c-4d9a-445b-a83b-66ddb514c09e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:48.463206 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:48.474814 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338282c-4d9a-445b-a83b-66ddb514c09e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-4338282c-4d9a-445b-a83b-66ddb514c09e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:48.480119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:48.480944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:35:48.481680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:48.483946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 56/102] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:35:48.494449 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338282c-4d9a-445b-a83b-66ddb514c09e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:48.520779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890c8e-ccf6-4376-8dc3-05d28ef1880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:48.521678 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890c8e-ccf6-4376-8dc3-05d28ef1880f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:48.523280 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890c8e-ccf6-4376-8dc3-05d28ef1880f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:48.523724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890c8e-ccf6-4376-8dc3-05d28ef1880f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:48.525680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37890c8e-ccf6-4376-8dc3-05d28ef1880f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:48.527480 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37890c8e-ccf6-4376-8dc3-05d28ef1880f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:48.530654 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 47/103] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:48.537854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890c8e-ccf6-4376-8dc3-05d28ef1880f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:48.597941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:48.659895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338282c-4d9a-445b-a83b-66ddb514c09e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:48.695069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b33db1-0001-4a88-b5c0-0353a3f75ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:48.696061 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b33db1-0001-4a88-b5c0-0353a3f75ce5 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:48.696536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b33db1-0001-4a88-b5c0-0353a3f75ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:48.697025 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b33db1-0001-4a88-b5c0-0353a3f75ce5 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:48.697899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b33db1-0001-4a88-b5c0-0353a3f75ce5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:48.698414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b33db1-0001-4a88-b5c0-0353a3f75ce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:48.700049 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b33db1-0001-4a88-b5c0-0353a3f75ce5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:48.709028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 57/104] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:35:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:48.710525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c7bb6-b4b2-4b1c-98f6-01c7b94830f8 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:48.715613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c7bb6-b4b2-4b1c-98f6-01c7b94830f8 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:48.716112 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c7bb6-b4b2-4b1c-98f6-01c7b94830f8 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:48.717620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 58/105] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:48.718525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c7bb6-b4b2-4b1c-98f6-01c7b94830f8 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:48.741482 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890c8e-ccf6-4376-8dc3-05d28ef1880f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:48.797962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c45966-9653-492d-a1ad-7ccfad7ee24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:48.798567 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c45966-9653-492d-a1ad-7ccfad7ee24a None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:48.798754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c45966-9653-492d-a1ad-7ccfad7ee24a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:48.799202 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c45966-9653-492d-a1ad-7ccfad7ee24a None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:48.799988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c45966-9653-492d-a1ad-7ccfad7ee24a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:48.801696 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c45966-9653-492d-a1ad-7ccfad7ee24a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:48.811322 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:48.859885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 48/106] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:35:48 2022] POST /identity/v3/projects => generated 302 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:35:48.861448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32fec7-2519-45fe-a74a-c5aaaab65669 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:48.861864 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32fec7-2519-45fe-a74a-c5aaaab65669 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:48.862294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32fec7-2519-45fe-a74a-c5aaaab65669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:48.891163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c7bb6-b4b2-4b1c-98f6-01c7b94830f8 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:48.901453 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-0a32fec7-2519-45fe-a74a-c5aaaab65669 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:48.902887 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-0a32fec7-2519-45fe-a74a-c5aaaab65669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:48.936445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd6607-7e5d-49db-8cd2-066980444530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:48.937388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd6607-7e5d-49db-8cd2-066980444530 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:48.937817 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd6607-7e5d-49db-8cd2-066980444530 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:48.938242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd6607-7e5d-49db-8cd2-066980444530 None admin] PATH_INFO: `/v3/users/432ed5207f9e45a1b59060f7f2bfac17` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:48.975300 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd6607-7e5d-49db-8cd2-066980444530 None admin] RBAC: Authorizing `identity:get_user(user_id=432ed5207f9e45a1b59060f7f2bfac17)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:48.977039 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd6607-7e5d-49db-8cd2-066980444530 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:48.980504 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 59/107] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:48 2022] GET /identity/v3/users/432ed5207f9e45a1b59060f7f2bfac17 => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:48.984747 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971af14-1a5b-40e9-a4d6-e28c6f14c944 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:48.987523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971af14-1a5b-40e9-a4d6-e28c6f14c944 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:48.987983 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971af14-1a5b-40e9-a4d6-e28c6f14c944 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:48.989968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 60/108] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:48.993005 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971af14-1a5b-40e9-a4d6-e28c6f14c944 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:49.030792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 49/109] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:49.033579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37965021-0cc2-41ee-aaaa-dbf9211befee None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:49.036508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37965021-0cc2-41ee-aaaa-dbf9211befee None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:49.036947 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37965021-0cc2-41ee-aaaa-dbf9211befee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:49.058426 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:49.061715 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-37965021-0cc2-41ee-aaaa-dbf9211befee None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:49.063334 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-37965021-0cc2-41ee-aaaa-dbf9211befee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:49.180703 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971af14-1a5b-40e9-a4d6-e28c6f14c944 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:49.207504 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:49.221462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd0934-1fb7-4100-b5c1-dbf0c58e9689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:49.222217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd0934-1fb7-4100-b5c1-dbf0c58e9689 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:49.222591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd0934-1fb7-4100-b5c1-dbf0c58e9689 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:49.223153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd0934-1fb7-4100-b5c1-dbf0c58e9689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:49.229520 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 50/110] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:49.342847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd0934-1fb7-4100-b5c1-dbf0c58e9689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:49.344296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd0934-1fb7-4100-b5c1-dbf0c58e9689 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:49.351071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37965021-0cc2-41ee-aaaa-dbf9211befee None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:49.520810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37965021-0cc2-41ee-aaaa-dbf9211befee None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:49.546833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 61/111] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:49.548242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0e8d7-9349-4426-b069-e5e146ab1846 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:49.553174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0e8d7-9349-4426-b069-e5e146ab1846 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:49.553566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0e8d7-9349-4426-b069-e5e146ab1846 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:49.558672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 62/112] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:49.558672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0e8d7-9349-4426-b069-e5e146ab1846 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:49.564211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf2c10-b05a-4894-aed5-aba6eb12e8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:49.565153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf2c10-b05a-4894-aed5-aba6eb12e8a0 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:49.565566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf2c10-b05a-4894-aed5-aba6eb12e8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:49.566671 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf2c10-b05a-4894-aed5-aba6eb12e8a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:49.571347 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:49.577164 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:49.586316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf2c10-b05a-4894-aed5-aba6eb12e8a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-f5bf2c10-b05a-4894-aed5-aba6eb12e8a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:49.591325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:49.592133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 51/113] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:35:49.593343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf2c10-b05a-4894-aed5-aba6eb12e8a0 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:49.685245 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0e8d7-9349-4426-b069-e5e146ab1846 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:49.703350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf2c10-b05a-4894-aed5-aba6eb12e8a0 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:49.722065 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919a38-b9b3-4a31-b9ce-2dd93db04753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:49.723002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919a38-b9b3-4a31-b9ce-2dd93db04753 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:49.723498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919a38-b9b3-4a31-b9ce-2dd93db04753 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:49.723947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919a38-b9b3-4a31-b9ce-2dd93db04753 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:49.734645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea919a38-b9b3-4a31-b9ce-2dd93db04753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:49.736931 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c029f4-c65f-4e45-9582-09c97353c560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:49.737569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c029f4-c65f-4e45-9582-09c97353c560 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:49.737849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c029f4-c65f-4e45-9582-09c97353c560 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:49.738120 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c029f4-c65f-4e45-9582-09c97353c560 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:49.739281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c029f4-c65f-4e45-9582-09c97353c560 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-ea919a38-b9b3-4a31-b9ce-2dd93db04753 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:49.740616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:49.741623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:49.742543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:49.743242 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c029f4-c65f-4e45-9582-09c97353c560 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:49.743242 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 52/114] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:49 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:49.744574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 63/115] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:35:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:49.745464 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919a38-b9b3-4a31-b9ce-2dd93db04753 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:49.752593 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10807fc3-d38c-4611-a966-aaffd00a05da None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:49.752876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10807fc3-d38c-4611-a966-aaffd00a05da None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:49.753152 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10807fc3-d38c-4611-a966-aaffd00a05da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:49.773532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-10807fc3-d38c-4611-a966-aaffd00a05da None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:49.774718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-10807fc3-d38c-4611-a966-aaffd00a05da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:49.824826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:49.833752 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:49.851220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 53/116] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:49.851220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89018c6-50a0-4162-a220-cebb1e68c38e None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:49.851220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89018c6-50a0-4162-a220-cebb1e68c38e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:49.851220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89018c6-50a0-4162-a220-cebb1e68c38e None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:49.851220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 54/117] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:49.851589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89018c6-50a0-4162-a220-cebb1e68c38e None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:49.858543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919a38-b9b3-4a31-b9ce-2dd93db04753 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:49.900430 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e06d74-4b16-4aab-a731-b7d3faae4c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:49.901057 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e06d74-4b16-4aab-a731-b7d3faae4c01 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:49.901360 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e06d74-4b16-4aab-a731-b7d3faae4c01 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:49.901635 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e06d74-4b16-4aab-a731-b7d3faae4c01 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:49.902308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e06d74-4b16-4aab-a731-b7d3faae4c01 None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:49.902609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e06d74-4b16-4aab-a731-b7d3faae4c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:49.904119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e06d74-4b16-4aab-a731-b7d3faae4c01 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:49.913638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 64/118] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:35:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:49.919037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e06d74-4b16-4aab-a731-b7d3faae4c01 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:49.973777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89018c6-50a0-4162-a220-cebb1e68c38e None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.008749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9589837-d511-4864-aa59-6975a0a5ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.009362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9589837-d511-4864-aa59-6975a0a5ac4a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.009639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9589837-d511-4864-aa59-6975a0a5ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.009908 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9589837-d511-4864-aa59-6975a0a5ac4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.010689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9589837-d511-4864-aa59-6975a0a5ac4a None admin] RBAC: Adding query filter params (role.id=1ece0be4f4c9430b9909bc8d8e87adc3, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224, user.id=432ed5207f9e45a1b59060f7f2bfac17) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:50.011000 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9589837-d511-4864-aa59-6975a0a5ac4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:50.012553 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9589837-d511-4864-aa59-6975a0a5ac4a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:50.018566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 55/119] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:49 2022] GET /identity/v3/role_assignments?user.id=432ed5207f9e45a1b59060f7f2bfac17&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=1ece0be4f4c9430b9909bc8d8e87adc3 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:50.026379 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fa9c2-96de-49bc-99f7-b3383c5dd2d5 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.026663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fa9c2-96de-49bc-99f7-b3383c5dd2d5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.026916 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fa9c2-96de-49bc-99f7-b3383c5dd2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.041770 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e06d74-4b16-4aab-a731-b7d3faae4c01 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.064343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-bd2fa9c2-96de-49bc-99f7-b3383c5dd2d5 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:50.065642 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-bd2fa9c2-96de-49bc-99f7-b3383c5dd2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:50.087091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72911fe0-b34c-4775-852a-710a0d2b8a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.088425 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72911fe0-b34c-4775-852a-710a0d2b8a2f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.088758 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72911fe0-b34c-4775-852a-710a0d2b8a2f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.089221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72911fe0-b34c-4775-852a-710a0d2b8a2f None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.090174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72911fe0-b34c-4775-852a-710a0d2b8a2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:50.090577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72911fe0-b34c-4775-852a-710a0d2b8a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:50.092503 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72911fe0-b34c-4775-852a-710a0d2b8a2f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:50.100706 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:50.106926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 65/120] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:50.107980 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72911fe0-b34c-4775-852a-710a0d2b8a2f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:50.164900 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:50.193524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 56/121] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:50.194479 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fa9c2-96de-49bc-99f7-b3383c5dd2d5 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:50.242186 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72911fe0-b34c-4775-852a-710a0d2b8a2f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.286559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53713-1513-48ca-8df4-ded3003e06bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.287436 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53713-1513-48ca-8df4-ded3003e06bf None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.287718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53713-1513-48ca-8df4-ded3003e06bf None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.288799 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53713-1513-48ca-8df4-ded3003e06bf None admin] PATH_INFO: `/v3/users/3092900ab6da4b199709df63140b22ea` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.290756 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c53713-1513-48ca-8df4-ded3003e06bf None admin] RBAC: Authorizing `identity:get_user(user_id=3092900ab6da4b199709df63140b22ea)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:50.292296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c53713-1513-48ca-8df4-ded3003e06bf None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:50.295623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 66/122] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:50 2022] GET /identity/v3/users/3092900ab6da4b199709df63140b22ea => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:50.296988 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53713-1513-48ca-8df4-ded3003e06bf None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:50.324575 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fa9c2-96de-49bc-99f7-b3383c5dd2d5 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.366300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ebde4-3b90-45d2-a13c-ff7e2026c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.366872 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ebde4-3b90-45d2-a13c-ff7e2026c33c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.367173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ebde4-3b90-45d2-a13c-ff7e2026c33c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.367447 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ebde4-3b90-45d2-a13c-ff7e2026c33c None admin] PATH_INFO: `/v3/users/506323730e1e4e22bdf37b533b4f458b` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.368694 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ebde4-3b90-45d2-a13c-ff7e2026c33c None admin] RBAC: Authorizing `identity:get_user(user_id=506323730e1e4e22bdf37b533b4f458b)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:50.370183 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ebde4-3b90-45d2-a13c-ff7e2026c33c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:50.371883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:50.372118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 57/123] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:50 2022] GET /identity/v3/users/506323730e1e4e22bdf37b533b4f458b => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:50.373243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ebde4-3b90-45d2-a13c-ff7e2026c33c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:50.412499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53713-1513-48ca-8df4-ded3003e06bf None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.463067 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:50.493569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8524d2a-5382-4d98-8329-e9e060f1248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.494217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8524d2a-5382-4d98-8329-e9e060f1248f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.494493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8524d2a-5382-4d98-8329-e9e060f1248f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.494763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8524d2a-5382-4d98-8329-e9e060f1248f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.514493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8524d2a-5382-4d98-8329-e9e060f1248f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-c8524d2a-5382-4d98-8329-e9e060f1248f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:50.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:50.520545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:50.521295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 67/124] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:35:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:50.521991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8524d2a-5382-4d98-8329-e9e060f1248f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:50.553385 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ebde4-3b90-45d2-a13c-ff7e2026c33c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.594809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd26ff6-b32a-4b94-94b3-0ce54bb933bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.595815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd26ff6-b32a-4b94-94b3-0ce54bb933bc None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.596546 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd26ff6-b32a-4b94-94b3-0ce54bb933bc None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.596885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd26ff6-b32a-4b94-94b3-0ce54bb933bc None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.598517 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd26ff6-b32a-4b94-94b3-0ce54bb933bc None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:50.600732 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd26ff6-b32a-4b94-94b3-0ce54bb933bc None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:50.602564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 58/125] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:50 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:50.603468 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd26ff6-b32a-4b94-94b3-0ce54bb933bc None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:50.647059 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8524d2a-5382-4d98-8329-e9e060f1248f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.675541 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:50.691547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb1f5c-370e-4982-a9ea-422422818a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.692458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb1f5c-370e-4982-a9ea-422422818a84 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.692738 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb1f5c-370e-4982-a9ea-422422818a84 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.693012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb1f5c-370e-4982-a9ea-422422818a84 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.703161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb1f5c-370e-4982-a9ea-422422818a84 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:50.703161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb1f5c-370e-4982-a9ea-422422818a84 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:50.703161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 68/126] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:50 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:50.703161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb1f5c-370e-4982-a9ea-422422818a84 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:50.739407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd26ff6-b32a-4b94-94b3-0ce54bb933bc None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.766856 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:50.782174 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56986f19-102a-4eab-a734-e22c5d08c710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.783285 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56986f19-102a-4eab-a734-e22c5d08c710 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.784269 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56986f19-102a-4eab-a734-e22c5d08c710 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.784682 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56986f19-102a-4eab-a734-e22c5d08c710 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.785750 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56986f19-102a-4eab-a734-e22c5d08c710 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:50.786188 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56986f19-102a-4eab-a734-e22c5d08c710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:50.788899 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56986f19-102a-4eab-a734-e22c5d08c710 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:50.799885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 59/127] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:35:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:50.801130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56986f19-102a-4eab-a734-e22c5d08c710 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:50.818766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb1f5c-370e-4982-a9ea-422422818a84 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.854577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137d722-9661-4c98-ab72-02bfffdb06dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.855274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d722-9661-4c98-ab72-02bfffdb06dd None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.855565 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d722-9661-4c98-ab72-02bfffdb06dd None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.855931 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d722-9661-4c98-ab72-02bfffdb06dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.871086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137d722-9661-4c98-ab72-02bfffdb06dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-1137d722-9661-4c98-ab72-02bfffdb06dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:50.877641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:50.878499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:50.879301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 69/128] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:50.883676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137d722-9661-4c98-ab72-02bfffdb06dd None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:50.920509 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56986f19-102a-4eab-a734-e22c5d08c710 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:50.960622 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb8455-3b2c-4f98-a559-e0724f63008a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:50.961239 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:50.962779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb8455-3b2c-4f98-a559-e0724f63008a None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:50.963299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb8455-3b2c-4f98-a559-e0724f63008a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:50.963886 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb8455-3b2c-4f98-a559-e0724f63008a None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/users/3092900ab6da4b199709df63140b22ea/roles/91e10f3d9d934fe1be9d3642b8092d08` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:50.966107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb8455-3b2c-4f98-a559-e0724f63008a None admin] RBAC: Authorizing `identity:create_grant(project_id=de3741f1b798440b9e1a4f4b24dbd224, user_id=3092900ab6da4b199709df63140b22ea, role_id=91e10f3d9d934fe1be9d3642b8092d08)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:50.968661 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb8455-3b2c-4f98-a559-e0724f63008a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:50.983514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 60/129] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:35:50 2022] PUT /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/users/3092900ab6da4b199709df63140b22ea/roles/91e10f3d9d934fe1be9d3642b8092d08 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:35:50.984784 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb8455-3b2c-4f98-a559-e0724f63008a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:51.027661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137d722-9661-4c98-ab72-02bfffdb06dd None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:51.037011 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:51.097325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e2a36-824e-4dc8-b161-0a7f01ad4c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:51.098018 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e2a36-824e-4dc8-b161-0a7f01ad4c12 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.098289 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e2a36-824e-4dc8-b161-0a7f01ad4c12 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.098566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e2a36-824e-4dc8-b161-0a7f01ad4c12 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/users/506323730e1e4e22bdf37b533b4f458b/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.100093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb8455-3b2c-4f98-a559-e0724f63008a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:51.103411 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e2a36-824e-4dc8-b161-0a7f01ad4c12 None admin] RBAC: Authorizing `identity:create_grant(project_id=de3741f1b798440b9e1a4f4b24dbd224, user_id=506323730e1e4e22bdf37b533b4f458b, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:51.104659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e2a36-824e-4dc8-b161-0a7f01ad4c12 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:51.134156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 70/130] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:35:50 2022] PUT /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/users/506323730e1e4e22bdf37b533b4f458b/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:35:51.135085 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e2a36-824e-4dc8-b161-0a7f01ad4c12 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:51.143612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f1fa0-ed16-42df-b5c3-c311275b6309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:51.144683 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f1fa0-ed16-42df-b5c3-c311275b6309 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.145174 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f1fa0-ed16-42df-b5c3-c311275b6309 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.145629 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f1fa0-ed16-42df-b5c3-c311275b6309 None admin] PATH_INFO: `/v3/users/3092900ab6da4b199709df63140b22ea` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.147458 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f1fa0-ed16-42df-b5c3-c311275b6309 None admin] RBAC: Authorizing `identity:get_user(user_id=3092900ab6da4b199709df63140b22ea)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:51.149299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f1fa0-ed16-42df-b5c3-c311275b6309 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:51.151300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 61/131] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:50 2022] GET /identity/v3/users/3092900ab6da4b199709df63140b22ea => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:51.157268 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f1fa0-ed16-42df-b5c3-c311275b6309 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:51.277560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f1fa0-ed16-42df-b5c3-c311275b6309 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:51.284316 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e2a36-824e-4dc8-b161-0a7f01ad4c12 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:51.292839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:51.302831 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:51.322128 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a15bb6-3045-446a-b3d1-978ca5eed320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:51.322895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a15bb6-3045-446a-b3d1-978ca5eed320 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.323199 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a15bb6-3045-446a-b3d1-978ca5eed320 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.323475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a15bb6-3045-446a-b3d1-978ca5eed320 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.323573 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e80f63-244a-41b2-9a32-d8ad343074fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:51.324151 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e80f63-244a-41b2-9a32-d8ad343074fc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.324440 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e80f63-244a-41b2-9a32-d8ad343074fc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.324669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a15bb6-3045-446a-b3d1-978ca5eed320 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:51.324764 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e80f63-244a-41b2-9a32-d8ad343074fc None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.325365 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e80f63-244a-41b2-9a32-d8ad343074fc None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:51.325665 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e80f63-244a-41b2-9a32-d8ad343074fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:51.326154 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a15bb6-3045-446a-b3d1-978ca5eed320 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:51.327060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e80f63-244a-41b2-9a32-d8ad343074fc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:51.328200 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 62/132] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:51 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:51.329790 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef2d03-f08c-4867-aac0-9c2bb0cf9455 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.330069 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef2d03-f08c-4867-aac0-9c2bb0cf9455 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.330337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef2d03-f08c-4867-aac0-9c2bb0cf9455 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.331589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 63/133] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:51.337056 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dcf80-b5be-44b0-a142-3944bf432642 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.337340 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dcf80-b5be-44b0-a142-3944bf432642 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.337614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dcf80-b5be-44b0-a142-3944bf432642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.342913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 71/134] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:35:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:51.344207 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e80f63-244a-41b2-9a32-d8ad343074fc None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:51.357314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-c95dcf80-b5be-44b0-a142-3944bf432642 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:51.358508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c95dcf80-b5be-44b0-a142-3944bf432642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:51.451109 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e80f63-244a-41b2-9a32-d8ad343074fc None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:51.452910 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 64/135] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:35:51.454102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dcf80-b5be-44b0-a142-3944bf432642 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:51.563749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dcf80-b5be-44b0-a142-3944bf432642 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:51.575349 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:51.579837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe9327-6bc5-4de9-bf21-3b521e54d7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:51.580531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe9327-6bc5-4de9-bf21-3b521e54d7d5 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.580807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe9327-6bc5-4de9-bf21-3b521e54d7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.581080 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe9327-6bc5-4de9-bf21-3b521e54d7d5 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/users/3092900ab6da4b199709df63140b22ea/roles/05aee5278bd743c4bb8fb605feb806a9` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.587877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe9327-6bc5-4de9-bf21-3b521e54d7d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=de3741f1b798440b9e1a4f4b24dbd224, user_id=3092900ab6da4b199709df63140b22ea, role_id=05aee5278bd743c4bb8fb605feb806a9)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:51.589459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe9327-6bc5-4de9-bf21-3b521e54d7d5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:51.602988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4db0e-75ce-4c7e-b083-0814bdc267de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:51.603773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4db0e-75ce-4c7e-b083-0814bdc267de None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.604052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4db0e-75ce-4c7e-b083-0814bdc267de None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.604322 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4db0e-75ce-4c7e-b083-0814bdc267de None admin] PATH_INFO: `/v3/users/3092900ab6da4b199709df63140b22ea` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.605444 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4db0e-75ce-4c7e-b083-0814bdc267de None admin] RBAC: Authorizing `identity:get_user(user_id=3092900ab6da4b199709df63140b22ea)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:51.607005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4db0e-75ce-4c7e-b083-0814bdc267de None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:51.607245 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 72/136] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:35:51 2022] PUT /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/users/3092900ab6da4b199709df63140b22ea/roles/05aee5278bd743c4bb8fb605feb806a9 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:35:51.608740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 65/137] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:51 2022] GET /identity/v3/users/3092900ab6da4b199709df63140b22ea => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:51.614546 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe9327-6bc5-4de9-bf21-3b521e54d7d5 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:51.683521 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:51.731338 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe9327-6bc5-4de9-bf21-3b521e54d7d5 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:51.768525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b25e3-1d5b-400d-afe9-acb2277e6eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:51.769270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b25e3-1d5b-400d-afe9-acb2277e6eb4 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.769634 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b25e3-1d5b-400d-afe9-acb2277e6eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.769964 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b25e3-1d5b-400d-afe9-acb2277e6eb4 None admin] PATH_INFO: `/v3/projects/c5fd9e09c605438eb4b2fb75ce124259` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.771266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b25e3-1d5b-400d-afe9-acb2277e6eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=c5fd9e09c605438eb4b2fb75ce124259)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:51.772898 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b25e3-1d5b-400d-afe9-acb2277e6eb4 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:51.775079 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 73/138] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:51 2022] GET /identity/v3/projects/c5fd9e09c605438eb4b2fb75ce124259 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:51.780840 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4db0e-75ce-4c7e-b083-0814bdc267de None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:51.898166 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4db0e-75ce-4c7e-b083-0814bdc267de None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:51.928946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b25e3-1d5b-400d-afe9-acb2277e6eb4 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:51.942532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e732e01-54c3-45d2-b876-3577545eba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:51.943312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e732e01-54c3-45d2-b876-3577545eba3b None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:51.943659 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e732e01-54c3-45d2-b876-3577545eba3b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:51.943953 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e732e01-54c3-45d2-b876-3577545eba3b None admin] PATH_INFO: `/v3/projects/c5fd9e09c605438eb4b2fb75ce124259/users/3092900ab6da4b199709df63140b22ea/roles/91e10f3d9d934fe1be9d3642b8092d08` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:51.948555 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e732e01-54c3-45d2-b876-3577545eba3b None admin] RBAC: Authorizing `identity:create_grant(project_id=c5fd9e09c605438eb4b2fb75ce124259, user_id=3092900ab6da4b199709df63140b22ea, role_id=91e10f3d9d934fe1be9d3642b8092d08)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:51.950292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e732e01-54c3-45d2-b876-3577545eba3b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:51.967780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 66/139] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:35:51 2022] PUT /identity/v3/projects/c5fd9e09c605438eb4b2fb75ce124259/users/3092900ab6da4b199709df63140b22ea/roles/91e10f3d9d934fe1be9d3642b8092d08 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:35:52.055949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b25e3-1d5b-400d-afe9-acb2277e6eb4 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:52.064949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:52.094787 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96409aca-54aa-4a56-acad-f16744b4a35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:52.095512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96409aca-54aa-4a56-acad-f16744b4a35f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:52.095789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96409aca-54aa-4a56-acad-f16744b4a35f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:52.096056 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96409aca-54aa-4a56-acad-f16744b4a35f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:52.097181 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96409aca-54aa-4a56-acad-f16744b4a35f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:52.098689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96409aca-54aa-4a56-acad-f16744b4a35f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:52.100756 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 74/140] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:52.106230 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e732e01-54c3-45d2-b876-3577545eba3b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:52.165601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:52.227768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e732e01-54c3-45d2-b876-3577545eba3b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:52.266619 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbed803-8fef-403d-bdd5-9eb66b6d3f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:52.267316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbed803-8fef-403d-bdd5-9eb66b6d3f5f None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:52.267599 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbed803-8fef-403d-bdd5-9eb66b6d3f5f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:52.267870 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbed803-8fef-403d-bdd5-9eb66b6d3f5f None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:52.268603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbed803-8fef-403d-bdd5-9eb66b6d3f5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:52.270067 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbed803-8fef-403d-bdd5-9eb66b6d3f5f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:52.301366 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 67/141] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:35:52 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:35:52.805308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afa95e-aced-4c78-a162-aae560f25c10 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:52.805995 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afa95e-aced-4c78-a162-aae560f25c10 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:52.806462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afa95e-aced-4c78-a162-aae560f25c10 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:52.808296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 75/142] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:52.813365 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e4f49-2746-47a7-8671-6f8d02f68ece None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:52.813885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e4f49-2746-47a7-8671-6f8d02f68ece None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:52.814346 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e4f49-2746-47a7-8671-6f8d02f68ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:52.836038 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-d60e4f49-2746-47a7-8671-6f8d02f68ece None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:52.837518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-d60e4f49-2746-47a7-8671-6f8d02f68ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:52.920917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 68/143] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:53.446557 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afa95e-aced-4c78-a162-aae560f25c10 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:53.566190 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afa95e-aced-4c78-a162-aae560f25c10 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:53.606606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5192f8-62b2-40a2-b8ef-0aacb089bae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:53.607676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5192f8-62b2-40a2-b8ef-0aacb089bae4 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:53.608140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5192f8-62b2-40a2-b8ef-0aacb089bae4 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:53.608588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5192f8-62b2-40a2-b8ef-0aacb089bae4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:53.624675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5192f8-62b2-40a2-b8ef-0aacb089bae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-2e5192f8-62b2-40a2-b8ef-0aacb089bae4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:53.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:53.630953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:35:53.631743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:53.634415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 76/144] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:53.639970 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e4f49-2746-47a7-8671-6f8d02f68ece None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:53.670599 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:53.725512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5b9cd-15d5-4a03-8110-a9344dc5ec38 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:53.726915 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5b9cd-15d5-4a03-8110-a9344dc5ec38 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:53.727404 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5b9cd-15d5-4a03-8110-a9344dc5ec38 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:53.729316 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 77/145] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:53.735239 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4804-f119-42b3-bb6a-fe340f072463 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:53.736143 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4804-f119-42b3-bb6a-fe340f072463 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:53.737427 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4804-f119-42b3-bb6a-fe340f072463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:53.770007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-0f2b4804-f119-42b3-bb6a-fe340f072463 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:53.771873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-0f2b4804-f119-42b3-bb6a-fe340f072463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:53.775889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e4f49-2746-47a7-8671-6f8d02f68ece None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:53.842435 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:53.850538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bcd43-06ba-49b2-861a-49c0cba59873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:53.851435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bcd43-06ba-49b2-861a-49c0cba59873 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:53.851720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bcd43-06ba-49b2-861a-49c0cba59873 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:53.851990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bcd43-06ba-49b2-861a-49c0cba59873 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:53.852668 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bcd43-06ba-49b2-861a-49c0cba59873 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:53.852968 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bcd43-06ba-49b2-861a-49c0cba59873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:53.854357 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bcd43-06ba-49b2-861a-49c0cba59873 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:53.874328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 69/146] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:35:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:53.880213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bcd43-06ba-49b2-861a-49c0cba59873 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:53.884000 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 78/147] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:54.001241 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:54.016010 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bcd43-06ba-49b2-861a-49c0cba59873 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:54.040695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac65872-4b83-439d-bd7a-ffe37a856bb5 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:54.041406 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac65872-4b83-439d-bd7a-ffe37a856bb5 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:54.041750 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac65872-4b83-439d-bd7a-ffe37a856bb5 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:54.043533 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 79/148] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:54.048620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa52a-1507-4e78-8a50-eb38526a8dc5 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:54.049133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa52a-1507-4e78-8a50-eb38526a8dc5 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:54.049598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa52a-1507-4e78-8a50-eb38526a8dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:54.055139 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f3978-f250-4eab-a26f-7d696d5ae934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:54.055139 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f3978-f250-4eab-a26f-7d696d5ae934 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:54.055349 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f3978-f250-4eab-a26f-7d696d5ae934 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:54.055620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f3978-f250-4eab-a26f-7d696d5ae934 None admin] PATH_INFO: `/v3/users/432ed5207f9e45a1b59060f7f2bfac17` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:54.056762 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f3978-f250-4eab-a26f-7d696d5ae934 None admin] RBAC: Authorizing `identity:get_user(user_id=432ed5207f9e45a1b59060f7f2bfac17)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:54.058213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f3978-f250-4eab-a26f-7d696d5ae934 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:54.059826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 70/149] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:53 2022] GET /identity/v3/users/432ed5207f9e45a1b59060f7f2bfac17 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:54.065394 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f3978-f250-4eab-a26f-7d696d5ae934 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:54.082045 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-8a7aa52a-1507-4e78-8a50-eb38526a8dc5 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:54.083407 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-8a7aa52a-1507-4e78-8a50-eb38526a8dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:54.237512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 80/150] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:35:54.262568 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f3978-f250-4eab-a26f-7d696d5ae934 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:54.308978 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c72ece-9d4d-4c71-a9ca-7b3c71b5aef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:54.309615 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c72ece-9d4d-4c71-a9ca-7b3c71b5aef6 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:54.309892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c72ece-9d4d-4c71-a9ca-7b3c71b5aef6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:54.310160 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c72ece-9d4d-4c71-a9ca-7b3c71b5aef6 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:54.311329 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c72ece-9d4d-4c71-a9ca-7b3c71b5aef6 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:54.312782 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c72ece-9d4d-4c71-a9ca-7b3c71b5aef6 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:54.314801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 71/151] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:54 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:54.320739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aa52a-1507-4e78-8a50-eb38526a8dc5 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:54.427629 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:54.434186 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aa52a-1507-4e78-8a50-eb38526a8dc5 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:54.486932 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415e8af-3c27-4431-9f78-fff1c4d92747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:54.487714 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415e8af-3c27-4431-9f78-fff1c4d92747 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:54.487897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415e8af-3c27-4431-9f78-fff1c4d92747 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:54.488164 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415e8af-3c27-4431-9f78-fff1c4d92747 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/users/432ed5207f9e45a1b59060f7f2bfac17/roles/1ece0be4f4c9430b9909bc8d8e87adc3` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:54.496568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415e8af-3c27-4431-9f78-fff1c4d92747 None admin] RBAC: Authorizing `identity:create_grant(project_id=de3741f1b798440b9e1a4f4b24dbd224, user_id=432ed5207f9e45a1b59060f7f2bfac17, role_id=1ece0be4f4c9430b9909bc8d8e87adc3)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:54.498048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415e8af-3c27-4431-9f78-fff1c4d92747 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:54.501759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c72ece-9d4d-4c71-a9ca-7b3c71b5aef6 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:54.525385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 81/152] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:35:54 2022] PUT /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/users/432ed5207f9e45a1b59060f7f2bfac17/roles/1ece0be4f4c9430b9909bc8d8e87adc3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:35:54.542401 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:54.585258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9d107-a0a7-4c11-8ede-cc293753fec6 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:54.585567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9d107-a0a7-4c11-8ede-cc293753fec6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:54.585797 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9d107-a0a7-4c11-8ede-cc293753fec6 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:54.587487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 82/153] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:35:54.595620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d813b7f-db86-44ef-b01b-1419c7425bb4 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:54.595891 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d813b7f-db86-44ef-b01b-1419c7425bb4 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:54.596161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d813b7f-db86-44ef-b01b-1419c7425bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:54.630620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-7d813b7f-db86-44ef-b01b-1419c7425bb4 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:54.632291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-7d813b7f-db86-44ef-b01b-1419c7425bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:54.640005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c72ece-9d4d-4c71-a9ca-7b3c71b5aef6 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:54.671224 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49be6f-0153-4819-8523-add3c616d742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:54.671862 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49be6f-0153-4819-8523-add3c616d742 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:54.672134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49be6f-0153-4819-8523-add3c616d742 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:54.672404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49be6f-0153-4819-8523-add3c616d742 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:54.682708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49be6f-0153-4819-8523-add3c616d742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-2c49be6f-0153-4819-8523-add3c616d742 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:54.688149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:54.689017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:35:54.689777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 72/154] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:35:54.716752 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49be6f-0153-4819-8523-add3c616d742 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:54.743911 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 83/155] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:54.832633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d813b7f-db86-44ef-b01b-1419c7425bb4 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:54.862343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49be6f-0153-4819-8523-add3c616d742 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:54.883412 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:54.920324 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb088-d9af-43c1-a12f-4508022c8b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:54.920945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb088-d9af-43c1-a12f-4508022c8b19 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:54.921219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb088-d9af-43c1-a12f-4508022c8b19 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:54.921483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb088-d9af-43c1-a12f-4508022c8b19 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:54.922163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeb088-d9af-43c1-a12f-4508022c8b19 None admin] RBAC: Adding query filter params (name=member) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:54.922466 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeb088-d9af-43c1-a12f-4508022c8b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:54.927948 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeb088-d9af-43c1-a12f-4508022c8b19 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:54.949754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 73/156] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:35:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:54.955998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb088-d9af-43c1-a12f-4508022c8b19 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:55.039499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:55.062979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d813b7f-db86-44ef-b01b-1419c7425bb4 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:55.072201 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb088-d9af-43c1-a12f-4508022c8b19 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:55.123782 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd74ee0-8b92-45ab-a4f3-679ca8f3ff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:55.124317 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd74ee0-8b92-45ab-a4f3-679ca8f3ff4e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:55.124591 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd74ee0-8b92-45ab-a4f3-679ca8f3ff4e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:55.124858 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd74ee0-8b92-45ab-a4f3-679ca8f3ff4e None admin] PATH_INFO: `/v3/users/3092900ab6da4b199709df63140b22ea` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:55.126024 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd74ee0-8b92-45ab-a4f3-679ca8f3ff4e None admin] RBAC: Authorizing `identity:get_user(user_id=3092900ab6da4b199709df63140b22ea)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:55.127507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd74ee0-8b92-45ab-a4f3-679ca8f3ff4e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:55.129269 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 74/157] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:54 2022] GET /identity/v3/users/3092900ab6da4b199709df63140b22ea => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:55.135823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd74ee0-8b92-45ab-a4f3-679ca8f3ff4e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:55.145938 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e279148-97dc-43c9-aec3-d96d80a0ad85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:55.146612 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279148-97dc-43c9-aec3-d96d80a0ad85 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:55.146989 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279148-97dc-43c9-aec3-d96d80a0ad85 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:55.147287 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279148-97dc-43c9-aec3-d96d80a0ad85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:55.155704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:55.179636 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e279148-97dc-43c9-aec3-d96d80a0ad85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-4e279148-97dc-43c9-aec3-d96d80a0ad85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:55.185133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:55.186217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:55.187388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 84/158] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:35:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:35:55.194273 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e279148-97dc-43c9-aec3-d96d80a0ad85 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:55.267620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd74ee0-8b92-45ab-a4f3-679ca8f3ff4e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:55.305990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83931384-ff7c-45b9-9438-7608f3811fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:55.306607 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83931384-ff7c-45b9-9438-7608f3811fcb None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:55.306882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83931384-ff7c-45b9-9438-7608f3811fcb None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:55.307179 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83931384-ff7c-45b9-9438-7608f3811fcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:55.392303 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e279148-97dc-43c9-aec3-d96d80a0ad85 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:55.453180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f59201-8cb1-42e4-be6a-528913c4dbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:55.454124 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f59201-8cb1-42e4-be6a-528913c4dbfe None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:55.454399 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f59201-8cb1-42e4-be6a-528913c4dbfe None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:55.454665 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f59201-8cb1-42e4-be6a-528913c4dbfe None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:55.455378 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f59201-8cb1-42e4-be6a-528913c4dbfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:55.455681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f59201-8cb1-42e4-be6a-528913c4dbfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:55.457165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f59201-8cb1-42e4-be6a-528913c4dbfe None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:55.466874 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 85/159] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:55.467568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f59201-8cb1-42e4-be6a-528913c4dbfe None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:55.483480 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:55.505299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83931384-ff7c-45b9-9438-7608f3811fcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:55.506820 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83931384-ff7c-45b9-9438-7608f3811fcb None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:55.542305 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:55.579512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f59201-8cb1-42e4-be6a-528913c4dbfe None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:55.620087 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c6636-28ab-4f36-85a8-03f8811ace1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:55.620740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c6636-28ab-4f36-85a8-03f8811ace1e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:55.621019 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c6636-28ab-4f36-85a8-03f8811ace1e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:55.621560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c6636-28ab-4f36-85a8-03f8811ace1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:55.634592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c6636-28ab-4f36-85a8-03f8811ace1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-070c6636-28ab-4f36-85a8-03f8811ace1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:55.640935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:55.641928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:55.642784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 86/160] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:35:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:55.643551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6af87-f1ec-4463-9d4d-89d2fb6118d2 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:55.644458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6af87-f1ec-4463-9d4d-89d2fb6118d2 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:55.644647 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6af87-f1ec-4463-9d4d-89d2fb6118d2 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:55.646083 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 87/161] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:55.646995 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6af87-f1ec-4463-9d4d-89d2fb6118d2 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:55.723577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:55.759893 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6af87-f1ec-4463-9d4d-89d2fb6118d2 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:55.796515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5127161-a036-4701-9452-2145e6e1207a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:55.797160 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5127161-a036-4701-9452-2145e6e1207a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:55.797444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5127161-a036-4701-9452-2145e6e1207a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:55.797720 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5127161-a036-4701-9452-2145e6e1207a None admin] PATH_INFO: `/v3/users/506323730e1e4e22bdf37b533b4f458b` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:55.798861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5127161-a036-4701-9452-2145e6e1207a None admin] RBAC: Authorizing `identity:get_user(user_id=506323730e1e4e22bdf37b533b4f458b)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:55.801249 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5127161-a036-4701-9452-2145e6e1207a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:55.803082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 88/162] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:55 2022] GET /identity/v3/users/506323730e1e4e22bdf37b533b4f458b => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:55.804054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81b03f-58a8-4325-bc42-48f1010723b7 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:55.804328 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81b03f-58a8-4325-bc42-48f1010723b7 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:55.804598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81b03f-58a8-4325-bc42-48f1010723b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:55.812479 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 75/163] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:35:55.813327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83931384-ff7c-45b9-9438-7608f3811fcb None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:55.851342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-8a81b03f-58a8-4325-bc42-48f1010723b7 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:55.852555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-8a81b03f-58a8-4325-bc42-48f1010723b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:55.995894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 89/164] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:55.996571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81b03f-58a8-4325-bc42-48f1010723b7 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:56.019057 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83931384-ff7c-45b9-9438-7608f3811fcb None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:56.048208 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:56.073549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:56.092421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b2c2d-01b3-4f94-bd67-e1fbedc25a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:56.093695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b2c2d-01b3-4f94-bd67-e1fbedc25a9e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:56.094182 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b2c2d-01b3-4f94-bd67-e1fbedc25a9e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:56.094718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b2c2d-01b3-4f94-bd67-e1fbedc25a9e None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:56.095777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b2c2d-01b3-4f94-bd67-e1fbedc25a9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:56.096289 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b2c2d-01b3-4f94-bd67-e1fbedc25a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:56.097996 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b2c2d-01b3-4f94-bd67-e1fbedc25a9e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:56.115960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81b03f-58a8-4325-bc42-48f1010723b7 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:56.118167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 76/165] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:35:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:56.119310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b2c2d-01b3-4f94-bd67-e1fbedc25a9e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:56.155463 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060e2f8-26df-4803-b85e-5a866c897914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:56.156080 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060e2f8-26df-4803-b85e-5a866c897914 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:56.156354 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060e2f8-26df-4803-b85e-5a866c897914 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:56.156623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060e2f8-26df-4803-b85e-5a866c897914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:56.237562 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060e2f8-26df-4803-b85e-5a866c897914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:56.239455 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060e2f8-26df-4803-b85e-5a866c897914 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:56.310268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:56.315780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b2c2d-01b3-4f94-bd67-e1fbedc25a9e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:56.354687 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 90/166] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:56.355671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060e2f8-26df-4803-b85e-5a866c897914 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:56.383730 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840578fe-af06-40cf-bccb-b3aa5a3d8201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:56.384836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840578fe-af06-40cf-bccb-b3aa5a3d8201 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:56.385321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840578fe-af06-40cf-bccb-b3aa5a3d8201 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:56.385795 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840578fe-af06-40cf-bccb-b3aa5a3d8201 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:56.387597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840578fe-af06-40cf-bccb-b3aa5a3d8201 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:56.389358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840578fe-af06-40cf-bccb-b3aa5a3d8201 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:56.395549 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 77/167] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:56 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:56.396633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840578fe-af06-40cf-bccb-b3aa5a3d8201 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:56.468798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060e2f8-26df-4803-b85e-5a866c897914 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:56.479144 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:56.509557 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce5139-381e-41b3-8cfa-0690370202a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:56.510202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce5139-381e-41b3-8cfa-0690370202a4 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:56.510478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce5139-381e-41b3-8cfa-0690370202a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:56.510742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce5139-381e-41b3-8cfa-0690370202a4 None admin] PATH_INFO: `/v3/users/3092900ab6da4b199709df63140b22ea` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:56.512344 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce5139-381e-41b3-8cfa-0690370202a4 None admin] RBAC: Authorizing `identity:get_user(user_id=3092900ab6da4b199709df63140b22ea)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:56.514083 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce5139-381e-41b3-8cfa-0690370202a4 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:56.515958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 91/168] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:56 2022] GET /identity/v3/users/3092900ab6da4b199709df63140b22ea => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:56.516811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce5139-381e-41b3-8cfa-0690370202a4 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:56.554334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:56.585598 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840578fe-af06-40cf-bccb-b3aa5a3d8201 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:56.631752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce5139-381e-41b3-8cfa-0690370202a4 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:56.643821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cd237-be52-44e9-8a90-dcc2b0a7c9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:56.644464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cd237-be52-44e9-8a90-dcc2b0a7c9c1 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:56.644803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cd237-be52-44e9-8a90-dcc2b0a7c9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:56.645070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cd237-be52-44e9-8a90-dcc2b0a7c9c1 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:56.646225 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cd237-be52-44e9-8a90-dcc2b0a7c9c1 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:56.647788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cd237-be52-44e9-8a90-dcc2b0a7c9c1 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:56.649439 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 78/169] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:56 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:56.650198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cd237-be52-44e9-8a90-dcc2b0a7c9c1 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:56.672387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfefb65-315b-48e5-963e-c224787a2874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:56.673008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfefb65-315b-48e5-963e-c224787a2874 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:56.673281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfefb65-315b-48e5-963e-c224787a2874 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:56.673546 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfefb65-315b-48e5-963e-c224787a2874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:56.674277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfefb65-315b-48e5-963e-c224787a2874 None admin] RBAC: Adding query filter params (role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224, user.id=3092900ab6da4b199709df63140b22ea) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:56.674577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfefb65-315b-48e5-963e-c224787a2874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:56.676052 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfefb65-315b-48e5-963e-c224787a2874 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:56.690218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 92/170] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:56 2022] GET /identity/v3/role_assignments?user.id=3092900ab6da4b199709df63140b22ea&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:56.691058 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfefb65-315b-48e5-963e-c224787a2874 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:56.835881 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:56.839793 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfefb65-315b-48e5-963e-c224787a2874 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:56.841991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cd237-be52-44e9-8a90-dcc2b0a7c9c1 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:56.859544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:56.878564 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90061a7a-4faf-44d0-89ec-ab29bcc529e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:56.879317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90061a7a-4faf-44d0-89ec-ab29bcc529e5 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:56.879657 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90061a7a-4faf-44d0-89ec-ab29bcc529e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:56.879928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90061a7a-4faf-44d0-89ec-ab29bcc529e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:56.892317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90061a7a-4faf-44d0-89ec-ab29bcc529e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-90061a7a-4faf-44d0-89ec-ab29bcc529e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:56.898084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:56.899088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:56.899902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 93/171] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:56.900566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90061a7a-4faf-44d0-89ec-ab29bcc529e5 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:56.910934 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b450fce-83e7-4c51-822c-7a16beb8d470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:56.912040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b450fce-83e7-4c51-822c-7a16beb8d470 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:56.912608 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b450fce-83e7-4c51-822c-7a16beb8d470 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:56.913079 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b450fce-83e7-4c51-822c-7a16beb8d470 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.011076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90061a7a-4faf-44d0-89ec-ab29bcc529e5 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:57.040858 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b450fce-83e7-4c51-822c-7a16beb8d470 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:57.042823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b450fce-83e7-4c51-822c-7a16beb8d470 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:57.049093 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158ebe-e86e-4c7e-a8c8-3fecff713273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:57.050040 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158ebe-e86e-4c7e-a8c8-3fecff713273 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.050533 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158ebe-e86e-4c7e-a8c8-3fecff713273 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.050996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158ebe-e86e-4c7e-a8c8-3fecff713273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.052046 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e158ebe-e86e-4c7e-a8c8-3fecff713273 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224, user.id=506323730e1e4e22bdf37b533b4f458b) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:57.052682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e158ebe-e86e-4c7e-a8c8-3fecff713273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:57.054381 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e158ebe-e86e-4c7e-a8c8-3fecff713273 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:57.061085 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 94/172] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:56 2022] GET /identity/v3/role_assignments?user.id=506323730e1e4e22bdf37b533b4f458b&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:57.062600 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fd2d2-c93d-4e0e-9e2b-ca3b79237526 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.063085 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fd2d2-c93d-4e0e-9e2b-ca3b79237526 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.063578 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fd2d2-c93d-4e0e-9e2b-ca3b79237526 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.065153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 95/173] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:57.066143 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fd2d2-c93d-4e0e-9e2b-ca3b79237526 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:57.092607 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:57.184540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fd2d2-c93d-4e0e-9e2b-ca3b79237526 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:57.223139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02834402-9fd0-4df5-a78b-76d9906bcf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:57.223954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02834402-9fd0-4df5-a78b-76d9906bcf94 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.224333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02834402-9fd0-4df5-a78b-76d9906bcf94 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.224745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02834402-9fd0-4df5-a78b-76d9906bcf94 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.225551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02834402-9fd0-4df5-a78b-76d9906bcf94 None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:57.225957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02834402-9fd0-4df5-a78b-76d9906bcf94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:57.227566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 79/174] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:57.227925 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02834402-9fd0-4df5-a78b-76d9906bcf94 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:57.229165 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c4934-60b3-4515-83da-cf0c9fbc29d4 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.229697 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c4934-60b3-4515-83da-cf0c9fbc29d4 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.230115 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c4934-60b3-4515-83da-cf0c9fbc29d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.238306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 96/175] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:35:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:57.239314 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02834402-9fd0-4df5-a78b-76d9906bcf94 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:57.264284 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-897c4934-60b3-4515-83da-cf0c9fbc29d4 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:57.265673 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-897c4934-60b3-4515-83da-cf0c9fbc29d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:57.268830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:57.336156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:57.352434 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02834402-9fd0-4df5-a78b-76d9906bcf94 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:57.406060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90b93b-fcf8-4fa5-b07f-d9ac395445aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:57.406721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90b93b-fcf8-4fa5-b07f-d9ac395445aa None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.406993 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90b93b-fcf8-4fa5-b07f-d9ac395445aa None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.407289 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90b93b-fcf8-4fa5-b07f-d9ac395445aa None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.413307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90b93b-fcf8-4fa5-b07f-d9ac395445aa None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:57.415003 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90b93b-fcf8-4fa5-b07f-d9ac395445aa None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:57.416555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 97/176] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:57 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:57.417332 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90b93b-fcf8-4fa5-b07f-d9ac395445aa None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:57.419663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 80/177] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:57.421383 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fb5de-902c-4ff4-814f-53abea092ba7 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.421978 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fb5de-902c-4ff4-814f-53abea092ba7 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.422478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fb5de-902c-4ff4-814f-53abea092ba7 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.424010 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 81/178] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:57.424903 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fb5de-902c-4ff4-814f-53abea092ba7 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:57.541983 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90b93b-fcf8-4fa5-b07f-d9ac395445aa None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:57.556581 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fb5de-902c-4ff4-814f-53abea092ba7 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:57.579827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5042ec-f762-481b-b475-a0f0aaf13d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:57.580617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5042ec-f762-481b-b475-a0f0aaf13d59 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.580987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5042ec-f762-481b-b475-a0f0aaf13d59 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.581474 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5042ec-f762-481b-b475-a0f0aaf13d59 None admin] PATH_INFO: `/v3/users/3092900ab6da4b199709df63140b22ea` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.582742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5042ec-f762-481b-b475-a0f0aaf13d59 None admin] RBAC: Authorizing `identity:get_user(user_id=3092900ab6da4b199709df63140b22ea)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:57.584536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5042ec-f762-481b-b475-a0f0aaf13d59 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:57.586329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 98/179] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:57 2022] GET /identity/v3/users/3092900ab6da4b199709df63140b22ea => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:57.587602 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca926a-7730-4aca-9c91-01fc1fb78083 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.589010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca926a-7730-4aca-9c91-01fc1fb78083 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.589010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca926a-7730-4aca-9c91-01fc1fb78083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.600032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1aac7-a89c-4a4a-835a-0d7134d8f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:57.600939 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1aac7-a89c-4a4a-835a-0d7134d8f455 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.601388 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1aac7-a89c-4a4a-835a-0d7134d8f455 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.601823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1aac7-a89c-4a4a-835a-0d7134d8f455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.602799 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1aac7-a89c-4a4a-835a-0d7134d8f455 None admin] RBAC: Adding query filter params (role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224, user.id=3092900ab6da4b199709df63140b22ea) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:57.603756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1aac7-a89c-4a4a-835a-0d7134d8f455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:57.605701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1aac7-a89c-4a4a-835a-0d7134d8f455 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:57.609182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-60ca926a-7730-4aca-9c91-01fc1fb78083 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:57.610523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-60ca926a-7730-4aca-9c91-01fc1fb78083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:57.614197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 82/180] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:57 2022] GET /identity/v3/role_assignments?user.id=3092900ab6da4b199709df63140b22ea&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:57.615346 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1aac7-a89c-4a4a-835a-0d7134d8f455 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:57.621017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:57.631306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:57.738286 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 99/181] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:57.739613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801bcf3-001e-4a3a-a3cf-ee09e2d29757 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.739892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801bcf3-001e-4a3a-a3cf-ee09e2d29757 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.740232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801bcf3-001e-4a3a-a3cf-ee09e2d29757 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.742880 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 100/182] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:57.749897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039ab62-2da5-4097-9625-1cadbd97917d None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.750197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039ab62-2da5-4097-9625-1cadbd97917d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.750438 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039ab62-2da5-4097-9625-1cadbd97917d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.790145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-4039ab62-2da5-4097-9625-1cadbd97917d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:57.791367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-4039ab62-2da5-4097-9625-1cadbd97917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:57.823213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1aac7-a89c-4a4a-835a-0d7134d8f455 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:57.864305 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 101/183] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:57.865153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039ab62-2da5-4097-9625-1cadbd97917d None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:57.866275 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09366-4b9e-46ef-8f39-89c65af4bb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:57.866943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09366-4b9e-46ef-8f39-89c65af4bb52 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:57.867190 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09366-4b9e-46ef-8f39-89c65af4bb52 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:57.867789 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09366-4b9e-46ef-8f39-89c65af4bb52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:57.965020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:57.966262 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09366-4b9e-46ef-8f39-89c65af4bb52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:57.968232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09366-4b9e-46ef-8f39-89c65af4bb52 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:57.972980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:57.993963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039ab62-2da5-4097-9625-1cadbd97917d None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.032955 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e9277-1622-4076-b447-5bf671ddf095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.033603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e9277-1622-4076-b447-5bf671ddf095 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.034051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e9277-1622-4076-b447-5bf671ddf095 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.034462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e9277-1622-4076-b447-5bf671ddf095 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.047534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e9277-1622-4076-b447-5bf671ddf095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-170e9277-1622-4076-b447-5bf671ddf095 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:58.053274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:58.054363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:35:58.055234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:58.057417 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 102/184] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:35:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:58.063869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e9277-1622-4076-b447-5bf671ddf095 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:58.108905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 83/185] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:58.114968 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09366-4b9e-46ef-8f39-89c65af4bb52 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:58.191929 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e9277-1622-4076-b447-5bf671ddf095 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.228526 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a721d-5716-4938-aaef-c9b2eb9d9587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.229432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a721d-5716-4938-aaef-c9b2eb9d9587 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.229864 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a721d-5716-4938-aaef-c9b2eb9d9587 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.230263 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a721d-5716-4938-aaef-c9b2eb9d9587 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.231167 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a721d-5716-4938-aaef-c9b2eb9d9587 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:58.231603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a721d-5716-4938-aaef-c9b2eb9d9587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:58.233237 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a721d-5716-4938-aaef-c9b2eb9d9587 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:58.239463 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:58.243079 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09366-4b9e-46ef-8f39-89c65af4bb52 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.243840 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 103/186] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:58.247614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:58.250576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a721d-5716-4938-aaef-c9b2eb9d9587 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:58.289856 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335fd5f-96f4-4bb3-a0f5-083f6cff406e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.290495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335fd5f-96f4-4bb3-a0f5-083f6cff406e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.290769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335fd5f-96f4-4bb3-a0f5-083f6cff406e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.291042 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335fd5f-96f4-4bb3-a0f5-083f6cff406e None admin] PATH_INFO: `/v3/projects/c5fd9e09c605438eb4b2fb75ce124259` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.292215 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335fd5f-96f4-4bb3-a0f5-083f6cff406e None admin] RBAC: Authorizing `identity:get_project(project_id=c5fd9e09c605438eb4b2fb75ce124259)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:58.293686 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335fd5f-96f4-4bb3-a0f5-083f6cff406e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:58.295375 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 84/187] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:58 2022] GET /identity/v3/projects/c5fd9e09c605438eb4b2fb75ce124259 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:58.296130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335fd5f-96f4-4bb3-a0f5-083f6cff406e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:58.361375 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a721d-5716-4938-aaef-c9b2eb9d9587 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.395059 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335fd5f-96f4-4bb3-a0f5-083f6cff406e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.395917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79441b0d-dc56-40d1-95f8-54afc947a6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.396579 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79441b0d-dc56-40d1-95f8-54afc947a6c9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.396847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79441b0d-dc56-40d1-95f8-54afc947a6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.397111 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79441b0d-dc56-40d1-95f8-54afc947a6c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.398213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79441b0d-dc56-40d1-95f8-54afc947a6c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:58.399690 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79441b0d-dc56-40d1-95f8-54afc947a6c9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:58.401259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 104/188] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:35:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:58.402718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79441b0d-dc56-40d1-95f8-54afc947a6c9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:58.430435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56688a2f-2832-4c79-b31d-469340cb45e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.431072 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56688a2f-2832-4c79-b31d-469340cb45e7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.431386 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56688a2f-2832-4c79-b31d-469340cb45e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.431655 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56688a2f-2832-4c79-b31d-469340cb45e7 None admin] PATH_INFO: `/v3/users/2b7a90d238674f3ba53b737c52d82da3` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.445421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56688a2f-2832-4c79-b31d-469340cb45e7 None admin] RBAC: Authorizing `identity:get_user(user_id=2b7a90d238674f3ba53b737c52d82da3)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:58.446940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56688a2f-2832-4c79-b31d-469340cb45e7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:58.448667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 85/189] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:58 2022] GET /identity/v3/users/2b7a90d238674f3ba53b737c52d82da3 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:58.451955 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56688a2f-2832-4c79-b31d-469340cb45e7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:58.469728 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:58.481663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:58.501357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79441b0d-dc56-40d1-95f8-54afc947a6c9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.533992 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5785c1f-15fa-41ef-a9aa-c2c13078abe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.534617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5785c1f-15fa-41ef-a9aa-c2c13078abe8 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.534885 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5785c1f-15fa-41ef-a9aa-c2c13078abe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.535176 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5785c1f-15fa-41ef-a9aa-c2c13078abe8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.545023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5785c1f-15fa-41ef-a9aa-c2c13078abe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-c5785c1f-15fa-41ef-a9aa-c2c13078abe8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:58.549829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:58.550688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:35:58.551731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:58.552335 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56688a2f-2832-4c79-b31d-469340cb45e7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.552413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 105/190] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:35:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:58.552413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee2a08-524f-4206-b761-479fe747f179 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.552539 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee2a08-524f-4206-b761-479fe747f179 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.552797 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee2a08-524f-4206-b761-479fe747f179 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.554564 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 106/191] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:35:58.556478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee2a08-524f-4206-b761-479fe747f179 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:58.587233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0823-e0b1-41e5-911e-d0c17f125f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.587842 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0823-e0b1-41e5-911e-d0c17f125f86 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.588119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0823-e0b1-41e5-911e-d0c17f125f86 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.588390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0823-e0b1-41e5-911e-d0c17f125f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.589151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb0823-e0b1-41e5-911e-d0c17f125f86 None admin] RBAC: Adding query filter params (role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=c5fd9e09c605438eb4b2fb75ce124259, user.id=3092900ab6da4b199709df63140b22ea) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:58.589430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb0823-e0b1-41e5-911e-d0c17f125f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:58.590867 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb0823-e0b1-41e5-911e-d0c17f125f86 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:58.596827 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 86/192] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:58 2022] GET /identity/v3/role_assignments?user.id=3092900ab6da4b199709df63140b22ea&scope.project.id=c5fd9e09c605438eb4b2fb75ce124259&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:58.597609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0823-e0b1-41e5-911e-d0c17f125f86 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:58.662920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee2a08-524f-4206-b761-479fe747f179 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.691936 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:58.699157 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:58.702386 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0609e3-43ef-4fe7-ae39-16d6ffa5d899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.703022 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0609e3-43ef-4fe7-ae39-16d6ffa5d899 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.703324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0609e3-43ef-4fe7-ae39-16d6ffa5d899 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.703591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0609e3-43ef-4fe7-ae39-16d6ffa5d899 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.704568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0609e3-43ef-4fe7-ae39-16d6ffa5d899 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:58.706050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0609e3-43ef-4fe7-ae39-16d6ffa5d899 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:58.708380 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0823-e0b1-41e5-911e-d0c17f125f86 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.739145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 107/193] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:35:58 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:35:58.739145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0609e3-43ef-4fe7-ae39-16d6ffa5d899 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:58.759906 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda5ae0-6099-46bf-9920-33f59242823a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.760534 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5ae0-6099-46bf-9920-33f59242823a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.760810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5ae0-6099-46bf-9920-33f59242823a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.761078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5ae0-6099-46bf-9920-33f59242823a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.849946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda5ae0-6099-46bf-9920-33f59242823a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:58.851569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda5ae0-6099-46bf-9920-33f59242823a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:58.872180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0609e3-43ef-4fe7-ae39-16d6ffa5d899 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:58.910995 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d1f33-9079-476f-a93c-9f29acfc7cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:58.911725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d1f33-9079-476f-a93c-9f29acfc7cd8 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.912081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d1f33-9079-476f-a93c-9f29acfc7cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.912330 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d1f33-9079-476f-a93c-9f29acfc7cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.913127 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d1f33-9079-476f-a93c-9f29acfc7cd8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:58.913367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d1f33-9079-476f-a93c-9f29acfc7cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:58.914810 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d1f33-9079-476f-a93c-9f29acfc7cd8 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:58.923786 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 108/194] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:35:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:58.924979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172d1b-7192-4c4f-8560-00d668114e57 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:58.925262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172d1b-7192-4c4f-8560-00d668114e57 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:58.925524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172d1b-7192-4c4f-8560-00d668114e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:58.946991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-f0172d1b-7192-4c4f-8560-00d668114e57 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:35:58.950189 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f0172d1b-7192-4c4f-8560-00d668114e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:35:58.969503 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:58.979144 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 87/195] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:35:58.980107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda5ae0-6099-46bf-9920-33f59242823a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:59.002555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:59.037706 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 109/196] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:35:59.037706 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0172d1b-7192-4c4f-8560-00d668114e57 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:59.099557 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda5ae0-6099-46bf-9920-33f59242823a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:59.138882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9b61b-b915-4c6e-a777-143d17ce93c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:59.139916 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9b61b-b915-4c6e-a777-143d17ce93c8 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:59.140470 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9b61b-b915-4c6e-a777-143d17ce93c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:59.140943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9b61b-b915-4c6e-a777-143d17ce93c8 None admin] PATH_INFO: `/v3/users/2b7a90d238674f3ba53b737c52d82da3` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:59.142398 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9b61b-b915-4c6e-a777-143d17ce93c8 None admin] RBAC: Authorizing `identity:get_user(user_id=2b7a90d238674f3ba53b737c52d82da3)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:59.144165 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9b61b-b915-4c6e-a777-143d17ce93c8 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:59.146337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 88/197] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:58 2022] GET /identity/v3/users/2b7a90d238674f3ba53b737c52d82da3 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:59.152253 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9b61b-b915-4c6e-a777-143d17ce93c8 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:59.162261 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0172d1b-7192-4c4f-8560-00d668114e57 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:59.207280 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df499324-bc66-4613-a06e-bf06e8f7bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:59.207890 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df499324-bc66-4613-a06e-bf06e8f7bd88 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:59.208165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df499324-bc66-4613-a06e-bf06e8f7bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:59.208431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df499324-bc66-4613-a06e-bf06e8f7bd88 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:59.209589 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df499324-bc66-4613-a06e-bf06e8f7bd88 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:59.211610 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df499324-bc66-4613-a06e-bf06e8f7bd88 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:59.213744 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 110/198] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:59 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:59.221563 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df499324-bc66-4613-a06e-bf06e8f7bd88 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:59.225102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:59.292243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9b61b-b915-4c6e-a777-143d17ce93c8 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:59.307810 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:59.345809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d61207-728d-435c-939a-cafefac6c976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:59.346680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d61207-728d-435c-939a-cafefac6c976 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:59.347097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d61207-728d-435c-939a-cafefac6c976 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:59.347561 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d61207-728d-435c-939a-cafefac6c976 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:59.388374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df499324-bc66-4613-a06e-bf06e8f7bd88 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:59.430727 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d61207-728d-435c-939a-cafefac6c976 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:59.432582 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d61207-728d-435c-939a-cafefac6c976 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:59.437243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db3770-be1c-4f0f-b7b3-f10ff637a813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:59.440667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db3770-be1c-4f0f-b7b3-f10ff637a813 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:59.441580 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db3770-be1c-4f0f-b7b3-f10ff637a813 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:59.442047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db3770-be1c-4f0f-b7b3-f10ff637a813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:59.443035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db3770-be1c-4f0f-b7b3-f10ff637a813 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=d7a7423471544bcb8851a5df381c6144, user.id=2b7a90d238674f3ba53b737c52d82da3) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:59.444010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db3770-be1c-4f0f-b7b3-f10ff637a813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:59.446238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db3770-be1c-4f0f-b7b3-f10ff637a813 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:59.454212 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 111/199] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:59 2022] GET /identity/v3/role_assignments?user.id=2b7a90d238674f3ba53b737c52d82da3&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:35:59.455358 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db3770-be1c-4f0f-b7b3-f10ff637a813 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:59.524815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:59.608179 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 89/200] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:35:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:35:59.613523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d61207-728d-435c-939a-cafefac6c976 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:59.644010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db3770-be1c-4f0f-b7b3-f10ff637a813 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:59.653864 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:59.681055 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57b89d-0a33-4955-8fa0-c2f5972ddc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:59.682104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57b89d-0a33-4955-8fa0-c2f5972ddc64 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:59.682578 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57b89d-0a33-4955-8fa0-c2f5972ddc64 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:59.683068 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57b89d-0a33-4955-8fa0-c2f5972ddc64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:59.694727 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57b89d-0a33-4955-8fa0-c2f5972ddc64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-7c57b89d-0a33-4955-8fa0-c2f5972ddc64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:35:59.700775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:35:59.701725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:35:59.702523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:35:59.705094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 112/201] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:35:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:35:59.712545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57b89d-0a33-4955-8fa0-c2f5972ddc64 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:59.777162 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d61207-728d-435c-939a-cafefac6c976 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:59.840548 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcec5a1-1c37-4171-b386-26ebc09bdc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:59.842326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcec5a1-1c37-4171-b386-26ebc09bdc25 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:59.847236 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcec5a1-1c37-4171-b386-26ebc09bdc25 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:59.847746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcec5a1-1c37-4171-b386-26ebc09bdc25 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:59.849173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcec5a1-1c37-4171-b386-26ebc09bdc25 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:59.850895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcec5a1-1c37-4171-b386-26ebc09bdc25 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:59.852817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 90/202] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:35:59 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:59.860699 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcec5a1-1c37-4171-b386-26ebc09bdc25 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:35:59.904763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57b89d-0a33-4955-8fa0-c2f5972ddc64 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:35:59.918325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:35:59.955611 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213bcc1-108c-4ea3-868d-8cbb0468c569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:35:59.956874 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213bcc1-108c-4ea3-868d-8cbb0468c569 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:35:59.957687 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213bcc1-108c-4ea3-868d-8cbb0468c569 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:35:59.958243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213bcc1-108c-4ea3-868d-8cbb0468c569 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:35:59.959183 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213bcc1-108c-4ea3-868d-8cbb0468c569 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:35:59.959682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213bcc1-108c-4ea3-868d-8cbb0468c569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:35:59.961322 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213bcc1-108c-4ea3-868d-8cbb0468c569 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:35:59.972580 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 113/203] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:35:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:35:59.978152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213bcc1-108c-4ea3-868d-8cbb0468c569 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:00.042336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:00.072050 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcec5a1-1c37-4171-b386-26ebc09bdc25 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:00.151642 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90adc49-db5d-407a-b136-448abde5d0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:00.152699 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90adc49-db5d-407a-b136-448abde5d0cc None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:00.153217 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90adc49-db5d-407a-b136-448abde5d0cc None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:00.153678 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90adc49-db5d-407a-b136-448abde5d0cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:00.154610 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90adc49-db5d-407a-b136-448abde5d0cc None admin] RBAC: Adding query filter params (role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=d7a7423471544bcb8851a5df381c6144, user.id=2b7a90d238674f3ba53b737c52d82da3) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:00.156344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90adc49-db5d-407a-b136-448abde5d0cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:00.158171 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90adc49-db5d-407a-b136-448abde5d0cc None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:00.172583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 91/204] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:35:59 2022] GET /identity/v3/role_assignments?user.id=2b7a90d238674f3ba53b737c52d82da3&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:00.178440 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213bcc1-108c-4ea3-868d-8cbb0468c569 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:00.236528 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8ba92-1747-4e8a-a8e9-7c47dec597d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:00.237548 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8ba92-1747-4e8a-a8e9-7c47dec597d8 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:00.238048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8ba92-1747-4e8a-a8e9-7c47dec597d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:00.238566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8ba92-1747-4e8a-a8e9-7c47dec597d8 None admin] PATH_INFO: `/v3/users/432ed5207f9e45a1b59060f7f2bfac17` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:00.240146 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8ba92-1747-4e8a-a8e9-7c47dec597d8 None admin] RBAC: Authorizing `identity:get_user(user_id=432ed5207f9e45a1b59060f7f2bfac17)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:00.241940 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8ba92-1747-4e8a-a8e9-7c47dec597d8 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:00.243928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 114/205] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:35:59 2022] GET /identity/v3/users/432ed5207f9e45a1b59060f7f2bfac17 => generated 289 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:00.249862 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90adc49-db5d-407a-b136-448abde5d0cc None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:00.357808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:00.366136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90adc49-db5d-407a-b136-448abde5d0cc None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:00.417950 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e9188-4957-4593-8a74-171523dac0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:00.418566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e9188-4957-4593-8a74-171523dac0dc None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:00.418839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e9188-4957-4593-8a74-171523dac0dc None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:00.419104 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e9188-4957-4593-8a74-171523dac0dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:00.495890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e9188-4957-4593-8a74-171523dac0dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:00.497367 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e9188-4957-4593-8a74-171523dac0dc None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:00.609232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:00.623109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 92/206] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:00.623309 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8ba92-1747-4e8a-a8e9-7c47dec597d8 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:00.751512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:00.758273 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8ba92-1747-4e8a-a8e9-7c47dec597d8 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:00.802596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3fa5c-db24-431c-a0e0-67e43f92122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:00.803671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3fa5c-db24-431c-a0e0-67e43f92122a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:00.804154 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3fa5c-db24-431c-a0e0-67e43f92122a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:00.804615 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3fa5c-db24-431c-a0e0-67e43f92122a None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:00.805990 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3fa5c-db24-431c-a0e0-67e43f92122a None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:00.807725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3fa5c-db24-431c-a0e0-67e43f92122a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:00.809615 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 115/207] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:00 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:00.815809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e9188-4957-4593-8a74-171523dac0dc None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:00.967028 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e9188-4957-4593-8a74-171523dac0dc None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:01.001345 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b34f06-ec31-4a46-8275-a5aa090ce6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:01.001951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b34f06-ec31-4a46-8275-a5aa090ce6e6 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.002226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b34f06-ec31-4a46-8275-a5aa090ce6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.002497 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b34f06-ec31-4a46-8275-a5aa090ce6e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.003259 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b34f06-ec31-4a46-8275-a5aa090ce6e6 None admin] RBAC: Adding query filter params (role.id=1ece0be4f4c9430b9909bc8d8e87adc3, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224, user.id=432ed5207f9e45a1b59060f7f2bfac17) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:01.003572 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b34f06-ec31-4a46-8275-a5aa090ce6e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:01.005026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b34f06-ec31-4a46-8275-a5aa090ce6e6 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:01.011313 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 93/208] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:00 2022] GET /identity/v3/role_assignments?user.id=432ed5207f9e45a1b59060f7f2bfac17&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=1ece0be4f4c9430b9909bc8d8e87adc3 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:01.244487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f7d0-1e3e-4f9d-b485-cbcb85f27f76 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.245011 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f7d0-1e3e-4f9d-b485-cbcb85f27f76 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.245011 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f7d0-1e3e-4f9d-b485-cbcb85f27f76 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.246616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 116/209] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:01.252031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c50d9f-efd9-4c5f-b2bf-b3977b77e2ba None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.252451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c50d9f-efd9-4c5f-b2bf-b3977b77e2ba None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.252582 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c50d9f-efd9-4c5f-b2bf-b3977b77e2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.274509 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-12c50d9f-efd9-4c5f-b2bf-b3977b77e2ba None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:01.275720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-12c50d9f-efd9-4c5f-b2bf-b3977b77e2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:01.282173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b3acf-7819-41d2-98fa-a16ee166b4f6 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.282450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b3acf-7819-41d2-98fa-a16ee166b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.282724 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b3acf-7819-41d2-98fa-a16ee166b4f6 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.284363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 117/210] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:01.290456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f96d8-5ec9-4b1e-ab0e-ffca70d2c95e None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.290737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f96d8-5ec9-4b1e-ab0e-ffca70d2c95e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.291002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f96d8-5ec9-4b1e-ab0e-ffca70d2c95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.317892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-dd2f96d8-5ec9-4b1e-ab0e-ffca70d2c95e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:01.319284 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-dd2f96d8-5ec9-4b1e-ab0e-ffca70d2c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:01.353756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:01.369336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 94/211] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:01.427289 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 118/212] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:01.653601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c50d9f-efd9-4c5f-b2bf-b3977b77e2ba None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:01.690637 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f96d8-5ec9-4b1e-ab0e-ffca70d2c95e None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:01.760931 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c50d9f-efd9-4c5f-b2bf-b3977b77e2ba None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:01.789165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:01.795543 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5abb77-ab00-4d85-8b4a-072f2fca1428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:01.795803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f96d8-5ec9-4b1e-ab0e-ffca70d2c95e None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:01.796186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5abb77-ab00-4d85-8b4a-072f2fca1428 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.796467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5abb77-ab00-4d85-8b4a-072f2fca1428 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.796736 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5abb77-ab00-4d85-8b4a-072f2fca1428 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.797850 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5abb77-ab00-4d85-8b4a-072f2fca1428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:01.799343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5abb77-ab00-4d85-8b4a-072f2fca1428 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:01.805632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 95/213] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:01.805632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c381c-9b3c-427c-8afa-f32c50b1d584 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.805632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c381c-9b3c-427c-8afa-f32c50b1d584 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.805632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c381c-9b3c-427c-8afa-f32c50b1d584 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.805632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 96/214] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:01.806722 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c381c-9b3c-427c-8afa-f32c50b1d584 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:01.835415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84a2cf-f2a0-472f-8f8a-773871d28f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:01.836089 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84a2cf-f2a0-472f-8f8a-773871d28f7f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.836361 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84a2cf-f2a0-472f-8f8a-773871d28f7f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.836625 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84a2cf-f2a0-472f-8f8a-773871d28f7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.847236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84a2cf-f2a0-472f-8f8a-773871d28f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-fd84a2cf-f2a0-472f-8f8a-773871d28f7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:01.852232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:01.853135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 119/215] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:01.855968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456dc4b-5f1d-4276-829c-fae2e8ad4676 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.856674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456dc4b-5f1d-4276-829c-fae2e8ad4676 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.856674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456dc4b-5f1d-4276-829c-fae2e8ad4676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.871373 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:01.876042 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-3456dc4b-5f1d-4276-829c-fae2e8ad4676 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:01.877695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-3456dc4b-5f1d-4276-829c-fae2e8ad4676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:01.920884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c381c-9b3c-427c-8afa-f32c50b1d584 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:01.947208 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 120/216] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:01.947208 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456dc4b-5f1d-4276-829c-fae2e8ad4676 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:01.969418 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e641b-82f8-4ca7-827c-04b75ee865e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:01.970161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e641b-82f8-4ca7-827c-04b75ee865e1 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:01.970570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e641b-82f8-4ca7-827c-04b75ee865e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:01.970907 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e641b-82f8-4ca7-827c-04b75ee865e1 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:01.971665 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e641b-82f8-4ca7-827c-04b75ee865e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:01.973307 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e641b-82f8-4ca7-827c-04b75ee865e1 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:01.999595 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 97/217] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:36:01 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:36:02.060462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:02.079372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456dc4b-5f1d-4276-829c-fae2e8ad4676 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:02.116952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd4f7-8a89-452c-adad-8eb1420cbd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:02.117586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd4f7-8a89-452c-adad-8eb1420cbd06 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:02.117864 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd4f7-8a89-452c-adad-8eb1420cbd06 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:02.118133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd4f7-8a89-452c-adad-8eb1420cbd06 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:02.118825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bd4f7-8a89-452c-adad-8eb1420cbd06 None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:02.119140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bd4f7-8a89-452c-adad-8eb1420cbd06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:02.120574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bd4f7-8a89-452c-adad-8eb1420cbd06 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:02.130577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 121/218] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:02.140347 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e641b-82f8-4ca7-827c-04b75ee865e1 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:02.231845 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd4f7-8a89-452c-adad-8eb1420cbd06 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:02.247289 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e641b-82f8-4ca7-827c-04b75ee865e1 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:02.259099 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:02.285878 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69297986-709c-4740-af15-e54a4860814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:02.286513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69297986-709c-4740-af15-e54a4860814e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:02.286784 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69297986-709c-4740-af15-e54a4860814e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:02.287092 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69297986-709c-4740-af15-e54a4860814e None admin] PATH_INFO: `/v3/users/476f725ae41944fc923b6419c2edf003` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:02.305800 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69297986-709c-4740-af15-e54a4860814e None admin] RBAC: Authorizing `identity:get_user(user_id=476f725ae41944fc923b6419c2edf003)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:02.307390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69297986-709c-4740-af15-e54a4860814e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:02.309136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 98/219] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:02 2022] GET /identity/v3/users/476f725ae41944fc923b6419c2edf003 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:02.310426 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cad45c-e1df-41cb-ac42-0ebebec61937 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:02.310583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cad45c-e1df-41cb-ac42-0ebebec61937 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:02.310856 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cad45c-e1df-41cb-ac42-0ebebec61937 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:02.312208 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 99/220] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:02.316494 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cad45c-e1df-41cb-ac42-0ebebec61937 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:02.356188 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd4f7-8a89-452c-adad-8eb1420cbd06 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:02.395695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02c889-8f19-4b22-8090-29d0d33e4ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:02.396840 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02c889-8f19-4b22-8090-29d0d33e4ca6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:02.397456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02c889-8f19-4b22-8090-29d0d33e4ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:02.397785 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02c889-8f19-4b22-8090-29d0d33e4ca6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:02.409356 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02c889-8f19-4b22-8090-29d0d33e4ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-9c02c889-8f19-4b22-8090-29d0d33e4ca6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:02.415307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:02.416223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:02.417012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 122/221] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:36:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:02.417965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508913-df29-4e70-889e-18ef6038b360 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:02.418198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508913-df29-4e70-889e-18ef6038b360 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:02.418428 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508913-df29-4e70-889e-18ef6038b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:02.436696 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cad45c-e1df-41cb-ac42-0ebebec61937 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:02.440546 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-27508913-df29-4e70-889e-18ef6038b360 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:02.441736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-27508913-df29-4e70-889e-18ef6038b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:02.444398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:02.480874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e61f5-c5fa-4a86-960b-b1f9c046c663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:02.481490 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e61f5-c5fa-4a86-960b-b1f9c046c663 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:02.481766 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e61f5-c5fa-4a86-960b-b1f9c046c663 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:02.482084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e61f5-c5fa-4a86-960b-b1f9c046c663 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:02.522682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 123/222] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:02.523740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27508913-df29-4e70-889e-18ef6038b360 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:02.534956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:02.574142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e61f5-c5fa-4a86-960b-b1f9c046c663 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:02.575719 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e61f5-c5fa-4a86-960b-b1f9c046c663 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:02.688798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27508913-df29-4e70-889e-18ef6038b360 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:02.753259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac35658-bbf9-4f1c-821c-dd03dff111f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:02.753807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac35658-bbf9-4f1c-821c-dd03dff111f0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:02.754077 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac35658-bbf9-4f1c-821c-dd03dff111f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:02.754336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac35658-bbf9-4f1c-821c-dd03dff111f0 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:02.755002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac35658-bbf9-4f1c-821c-dd03dff111f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:02.755317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac35658-bbf9-4f1c-821c-dd03dff111f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:02.756818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac35658-bbf9-4f1c-821c-dd03dff111f0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:02.763245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 100/223] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:02.767067 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:02.768928 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e61f5-c5fa-4a86-960b-b1f9c046c663 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:02.774820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 124/224] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:02.780330 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac35658-bbf9-4f1c-821c-dd03dff111f0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:02.876071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:02.922212 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e61f5-c5fa-4a86-960b-b1f9c046c663 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:02.922887 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac35658-bbf9-4f1c-821c-dd03dff111f0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:02.960394 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e9606-35f8-48e2-8533-0cdc2911c7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:02.961136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e9606-35f8-48e2-8533-0cdc2911c7ea None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:02.961415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e9606-35f8-48e2-8533-0cdc2911c7ea None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:02.961684 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e9606-35f8-48e2-8533-0cdc2911c7ea None admin] PATH_INFO: `/v3/users/2b7a90d238674f3ba53b737c52d82da3` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:02.962829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e9606-35f8-48e2-8533-0cdc2911c7ea None admin] RBAC: Authorizing `identity:get_user(user_id=2b7a90d238674f3ba53b737c52d82da3)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:02.964326 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e9606-35f8-48e2-8533-0cdc2911c7ea None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:02.965668 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbbcb2-75d7-47f6-aae8-bc104a44021c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:02.966133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 125/225] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:02 2022] GET /identity/v3/users/2b7a90d238674f3ba53b737c52d82da3 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:02.966431 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbbcb2-75d7-47f6-aae8-bc104a44021c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:02.966736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e9606-35f8-48e2-8533-0cdc2911c7ea None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:02.966985 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbbcb2-75d7-47f6-aae8-bc104a44021c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:02.967193 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbbcb2-75d7-47f6-aae8-bc104a44021c None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:02.968403 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbbcb2-75d7-47f6-aae8-bc104a44021c None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:02.970218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbbcb2-75d7-47f6-aae8-bc104a44021c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:02.972933 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 101/226] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:02 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:02.973692 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbbcb2-75d7-47f6-aae8-bc104a44021c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:03.057158 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:03.078230 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbbcb2-75d7-47f6-aae8-bc104a44021c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:03.099250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e9606-35f8-48e2-8533-0cdc2911c7ea None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:03.114902 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941bc2b-2739-4cdc-9c4a-7b647b809a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:03.115667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941bc2b-2739-4cdc-9c4a-7b647b809a04 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:03.115982 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941bc2b-2739-4cdc-9c4a-7b647b809a04 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:03.116423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941bc2b-2739-4cdc-9c4a-7b647b809a04 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:03.117670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941bc2b-2739-4cdc-9c4a-7b647b809a04 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:03.119223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941bc2b-2739-4cdc-9c4a-7b647b809a04 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:03.121067 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 102/227] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:02 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:03.121659 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941bc2b-2739-4cdc-9c4a-7b647b809a04 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:03.130472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:03.144115 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08623d5-51c7-4d34-89a3-12d987cfc5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:03.145050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08623d5-51c7-4d34-89a3-12d987cfc5b2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:03.145922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08623d5-51c7-4d34-89a3-12d987cfc5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:03.146468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08623d5-51c7-4d34-89a3-12d987cfc5b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:03.160599 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08623d5-51c7-4d34-89a3-12d987cfc5b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-a08623d5-51c7-4d34-89a3-12d987cfc5b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:03.168323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:03.169221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:03.170016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:03.177053 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 126/228] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:36:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:03.177053 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08623d5-51c7-4d34-89a3-12d987cfc5b2 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:03.246309 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941bc2b-2739-4cdc-9c4a-7b647b809a04 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:03.290235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08623d5-51c7-4d34-89a3-12d987cfc5b2 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:03.290579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6e520-bbc6-4078-9369-67a60f9ea422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:03.291082 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6e520-bbc6-4078-9369-67a60f9ea422 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:03.291389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6e520-bbc6-4078-9369-67a60f9ea422 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:03.291821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6e520-bbc6-4078-9369-67a60f9ea422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:03.293360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6e520-bbc6-4078-9369-67a60f9ea422 None admin] RBAC: Adding query filter params (role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=d7a7423471544bcb8851a5df381c6144, user.id=476f725ae41944fc923b6419c2edf003) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:03.293675 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6e520-bbc6-4078-9369-67a60f9ea422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:03.295145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6e520-bbc6-4078-9369-67a60f9ea422 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:03.302713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 103/229] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:03 2022] GET /identity/v3/role_assignments?user.id=476f725ae41944fc923b6419c2edf003&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:03.304161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6e520-bbc6-4078-9369-67a60f9ea422 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:03.328682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:03.340542 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de94b-023f-4e65-9f54-074a183a4152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:03.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de94b-023f-4e65-9f54-074a183a4152 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:03.341830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de94b-023f-4e65-9f54-074a183a4152 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:03.342108 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de94b-023f-4e65-9f54-074a183a4152 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144/users/2b7a90d238674f3ba53b737c52d82da3/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:03.345413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13de94b-023f-4e65-9f54-074a183a4152 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7a7423471544bcb8851a5df381c6144, user_id=2b7a90d238674f3ba53b737c52d82da3, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:03.346947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13de94b-023f-4e65-9f54-074a183a4152 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:03.366688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 127/230] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:36:03 2022] PUT /identity/v3/projects/d7a7423471544bcb8851a5df381c6144/users/2b7a90d238674f3ba53b737c52d82da3/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:03.424828 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:03.476541 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6e520-bbc6-4078-9369-67a60f9ea422 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:03.526094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb850c-fa0f-411b-93d9-f2c2d68982fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:03.526725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb850c-fa0f-411b-93d9-f2c2d68982fb None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:03.527005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb850c-fa0f-411b-93d9-f2c2d68982fb None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:03.527326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb850c-fa0f-411b-93d9-f2c2d68982fb None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:03.528018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb850c-fa0f-411b-93d9-f2c2d68982fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:03.528327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb850c-fa0f-411b-93d9-f2c2d68982fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:03.529765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb850c-fa0f-411b-93d9-f2c2d68982fb None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:03.541428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 104/231] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:03.547510 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de94b-023f-4e65-9f54-074a183a4152 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:03.677395 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de94b-023f-4e65-9f54-074a183a4152 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:03.713681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2744af8-d106-4679-8829-1664280d6377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:03.714392 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2744af8-d106-4679-8829-1664280d6377 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:03.714673 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2744af8-d106-4679-8829-1664280d6377 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:03.714945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2744af8-d106-4679-8829-1664280d6377 None admin] PATH_INFO: `/v3/users/2b7a90d238674f3ba53b737c52d82da3` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:03.716118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2744af8-d106-4679-8829-1664280d6377 None admin] RBAC: Authorizing `identity:get_user(user_id=2b7a90d238674f3ba53b737c52d82da3)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:03.717606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2744af8-d106-4679-8829-1664280d6377 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:03.719240 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 128/232] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:03 2022] GET /identity/v3/users/2b7a90d238674f3ba53b737c52d82da3 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:03.728318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb850c-fa0f-411b-93d9-f2c2d68982fb None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:03.848124 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb850c-fa0f-411b-93d9-f2c2d68982fb None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:03.887571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:03.893227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe46c0-0ec7-4301-b5f9-341fd42746a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:03.894222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe46c0-0ec7-4301-b5f9-341fd42746a7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:03.894754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe46c0-0ec7-4301-b5f9-341fd42746a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:03.895273 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe46c0-0ec7-4301-b5f9-341fd42746a7 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:03.900518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe46c0-0ec7-4301-b5f9-341fd42746a7 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:03.901970 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe46c0-0ec7-4301-b5f9-341fd42746a7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:03.903830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 105/233] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:03 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:03.909784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2744af8-d106-4679-8829-1664280d6377 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:03.994868 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:04.049538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2744af8-d106-4679-8829-1664280d6377 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:04.094540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985ead4-8f4f-4ac7-b89c-1dad4bd6b99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:04.095269 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985ead4-8f4f-4ac7-b89c-1dad4bd6b99c None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:04.095548 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985ead4-8f4f-4ac7-b89c-1dad4bd6b99c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:04.095819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985ead4-8f4f-4ac7-b89c-1dad4bd6b99c None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144/users/2b7a90d238674f3ba53b737c52d82da3/roles/05aee5278bd743c4bb8fb605feb806a9` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:04.098002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985ead4-8f4f-4ac7-b89c-1dad4bd6b99c None admin] RBAC: Authorizing `identity:create_grant(project_id=d7a7423471544bcb8851a5df381c6144, user_id=2b7a90d238674f3ba53b737c52d82da3, role_id=05aee5278bd743c4bb8fb605feb806a9)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:04.099577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985ead4-8f4f-4ac7-b89c-1dad4bd6b99c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:04.122680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 129/234] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:36:03 2022] PUT /identity/v3/projects/d7a7423471544bcb8851a5df381c6144/users/2b7a90d238674f3ba53b737c52d82da3/roles/05aee5278bd743c4bb8fb605feb806a9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:04.234782 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1e449-05ad-497e-ad48-4d74998d68cf None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:04.235105 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1e449-05ad-497e-ad48-4d74998d68cf None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:04.235354 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1e449-05ad-497e-ad48-4d74998d68cf None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:04.236769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 106/235] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:04.242232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d026df-14a7-4ffa-ab9a-7398267e564f None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:04.242550 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d026df-14a7-4ffa-ab9a-7398267e564f None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:04.242848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d026df-14a7-4ffa-ab9a-7398267e564f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:04.268791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-a2d026df-14a7-4ffa-ab9a-7398267e564f None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:04.269929 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-a2d026df-14a7-4ffa-ab9a-7398267e564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:04.357647 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 130/236] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:04.503358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b13da-f7ab-4318-bfa8-0d5c1874cad7 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:04.504304 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b13da-f7ab-4318-bfa8-0d5c1874cad7 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:04.504784 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b13da-f7ab-4318-bfa8-0d5c1874cad7 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:04.506405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 107/237] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:04.512175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bde22-d94c-438c-b90f-48646940fc5f None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:04.512592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bde22-d94c-438c-b90f-48646940fc5f None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:04.512759 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bde22-d94c-438c-b90f-48646940fc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:04.544204 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-d23bde22-d94c-438c-b90f-48646940fc5f None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:04.545309 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-d23bde22-d94c-438c-b90f-48646940fc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:04.592409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:04.619571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 131/238] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 30 11:36:04.670813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b13da-f7ab-4318-bfa8-0d5c1874cad7 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:04.807075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b13da-f7ab-4318-bfa8-0d5c1874cad7 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:04.859237 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ae96d-f4a4-42fa-b475-9174199e05fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:04.859887 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ae96d-f4a4-42fa-b475-9174199e05fb None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:04.860162 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ae96d-f4a4-42fa-b475-9174199e05fb None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:04.860430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ae96d-f4a4-42fa-b475-9174199e05fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:04.871975 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ae96d-f4a4-42fa-b475-9174199e05fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-5a5ae96d-f4a4-42fa-b475-9174199e05fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:04.876863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:04.877720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:04.878508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 108/239] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:04.888245 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bde22-d94c-438c-b90f-48646940fc5f None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:05.027663 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bde22-d94c-438c-b90f-48646940fc5f None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:05.063366 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884151f-9c6a-4917-b54e-138b505efa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:05.063905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884151f-9c6a-4917-b54e-138b505efa56 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.064178 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884151f-9c6a-4917-b54e-138b505efa56 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.064446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884151f-9c6a-4917-b54e-138b505efa56 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.065116 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884151f-9c6a-4917-b54e-138b505efa56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:05.065414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884151f-9c6a-4917-b54e-138b505efa56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:05.066807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884151f-9c6a-4917-b54e-138b505efa56 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:05.075460 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 132/240] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:05.081231 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ae96d-f4a4-42fa-b475-9174199e05fb None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:05.142640 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:05.153993 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884151f-9c6a-4917-b54e-138b505efa56 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:05.193222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ae96d-f4a4-42fa-b475-9174199e05fb None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:05.203468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:05.231703 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ed905-991c-4eb3-9719-f2d7141916f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:05.232598 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ed905-991c-4eb3-9719-f2d7141916f1 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.232882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ed905-991c-4eb3-9719-f2d7141916f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.233154 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ed905-991c-4eb3-9719-f2d7141916f1 None admin] PATH_INFO: `/v3/users/a1c7a935c2c84b259030d28b3449b9a3` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.249357 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ed905-991c-4eb3-9719-f2d7141916f1 None admin] RBAC: Authorizing `identity:get_user(user_id=a1c7a935c2c84b259030d28b3449b9a3)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:05.251181 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ed905-991c-4eb3-9719-f2d7141916f1 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:05.253051 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 109/241] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:05 2022] GET /identity/v3/users/a1c7a935c2c84b259030d28b3449b9a3 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:05.258451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ed905-991c-4eb3-9719-f2d7141916f1 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:05.353111 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884151f-9c6a-4917-b54e-138b505efa56 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:05.370296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ed905-991c-4eb3-9719-f2d7141916f1 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:05.389860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:05.421551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20151f-2cc2-4cbc-b721-ddbef9779e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:05.422214 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20151f-2cc2-4cbc-b721-ddbef9779e8d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.422488 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20151f-2cc2-4cbc-b721-ddbef9779e8d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.422754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20151f-2cc2-4cbc-b721-ddbef9779e8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.424650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20151f-2cc2-4cbc-b721-ddbef9779e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:05.426105 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20151f-2cc2-4cbc-b721-ddbef9779e8d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:05.431851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d35ff-2559-4a9e-a779-4cbc54d128e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:05.432473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d35ff-2559-4a9e-a779-4cbc54d128e1 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.432742 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d35ff-2559-4a9e-a779-4cbc54d128e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.433004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d35ff-2559-4a9e-a779-4cbc54d128e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.433538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 133/242] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:05.442606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20151f-2cc2-4cbc-b721-ddbef9779e8d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:05.523365 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d35ff-2559-4a9e-a779-4cbc54d128e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:05.525526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d35ff-2559-4a9e-a779-4cbc54d128e1 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:05.591434 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20151f-2cc2-4cbc-b721-ddbef9779e8d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:05.592114 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:05.621748 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dc460-adb1-4d83-95d8-1393d113f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:05.622326 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dc460-adb1-4d83-95d8-1393d113f1cb None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.622598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dc460-adb1-4d83-95d8-1393d113f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.622914 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dc460-adb1-4d83-95d8-1393d113f1cb None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.623925 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dc460-adb1-4d83-95d8-1393d113f1cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:05.625518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dc460-adb1-4d83-95d8-1393d113f1cb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:05.650385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 134/243] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:36:05 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:36:05.675518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c912d81-87cb-41bc-8c00-28ef93d32949 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.675831 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c912d81-87cb-41bc-8c00-28ef93d32949 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.676162 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c912d81-87cb-41bc-8c00-28ef93d32949 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.677895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 135/244] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:05.684040 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b93a34-ebe9-4b4c-b296-61c72e69ac22 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.684381 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b93a34-ebe9-4b4c-b296-61c72e69ac22 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.684633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b93a34-ebe9-4b4c-b296-61c72e69ac22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.708171 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-78b93a34-ebe9-4b4c-b296-61c72e69ac22 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:05.709432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-78b93a34-ebe9-4b4c-b296-61c72e69ac22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:05.759410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 110/245] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:36:05.766099 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d35ff-2559-4a9e-a779-4cbc54d128e1 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:05.800685 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:05.817882 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 136/246] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:05.823821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe646b8-0e3f-445f-af0c-082e60b65c2f None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.824186 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe646b8-0e3f-445f-af0c-082e60b65c2f None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.824465 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe646b8-0e3f-445f-af0c-082e60b65c2f None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.825838 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 137/247] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:05.831724 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17262d5-601e-4aa4-979d-4cb8fc63c37d None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.831997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17262d5-601e-4aa4-979d-4cb8fc63c37d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.832264 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17262d5-601e-4aa4-979d-4cb8fc63c37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.852496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-c17262d5-601e-4aa4-979d-4cb8fc63c37d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:05.853736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c17262d5-601e-4aa4-979d-4cb8fc63c37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:05.889286 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d35ff-2559-4a9e-a779-4cbc54d128e1 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:05.928395 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 138/248] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:05.930409 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc07f91-8182-447e-bcf4-a1d3b7bff9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:05.931040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc07f91-8182-447e-bcf4-a1d3b7bff9ac None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:05.931347 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc07f91-8182-447e-bcf4-a1d3b7bff9ac None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:05.931611 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc07f91-8182-447e-bcf4-a1d3b7bff9ac None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:05.932706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc07f91-8182-447e-bcf4-a1d3b7bff9ac None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:05.934154 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc07f91-8182-447e-bcf4-a1d3b7bff9ac None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:05.936033 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 111/249] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:05 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:05.941535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17262d5-601e-4aa4-979d-4cb8fc63c37d None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:05.968742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:06.044934 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17262d5-601e-4aa4-979d-4cb8fc63c37d None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:06.079078 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6569d5-4844-406e-ab58-b125a682b6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:06.079730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6569d5-4844-406e-ab58-b125a682b6a3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:06.079999 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6569d5-4844-406e-ab58-b125a682b6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:06.080258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6569d5-4844-406e-ab58-b125a682b6a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:06.080974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6569d5-4844-406e-ab58-b125a682b6a3 None admin] RBAC: Adding query filter params (role.id=1ece0be4f4c9430b9909bc8d8e87adc3, scope.project.id=d7a7423471544bcb8851a5df381c6144, user.id=a1c7a935c2c84b259030d28b3449b9a3) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:06.081278 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6569d5-4844-406e-ab58-b125a682b6a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:06.082739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6569d5-4844-406e-ab58-b125a682b6a3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:06.088699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 139/250] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:05 2022] GET /identity/v3/role_assignments?user.id=a1c7a935c2c84b259030d28b3449b9a3&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=1ece0be4f4c9430b9909bc8d8e87adc3 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:06.122759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc07f91-8182-447e-bcf4-a1d3b7bff9ac None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:06.218121 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:06.223843 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6569d5-4844-406e-ab58-b125a682b6a3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:06.243023 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc07f91-8182-447e-bcf4-a1d3b7bff9ac None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:06.285109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f6496-a8b9-45c9-8558-c82b56a1246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:06.285829 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f6496-a8b9-45c9-8558-c82b56a1246b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:06.288308 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f6496-a8b9-45c9-8558-c82b56a1246b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:06.288308 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f6496-a8b9-45c9-8558-c82b56a1246b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:06.301065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f6496-a8b9-45c9-8558-c82b56a1246b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-095f6496-a8b9-45c9-8558-c82b56a1246b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:06.306438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:06.307504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:06.308311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 112/251] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:06.318794 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f6496-a8b9-45c9-8558-c82b56a1246b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:06.343589 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:06.359401 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6569d5-4844-406e-ab58-b125a682b6a3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:06.404258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae69b2-6f5a-4f40-975c-c2fddc9234d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:06.404914 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae69b2-6f5a-4f40-975c-c2fddc9234d6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:06.405188 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae69b2-6f5a-4f40-975c-c2fddc9234d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:06.405451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae69b2-6f5a-4f40-975c-c2fddc9234d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:06.420674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae69b2-6f5a-4f40-975c-c2fddc9234d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-02ae69b2-6f5a-4f40-975c-c2fddc9234d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:06.426633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:06.427617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:06.428429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:06.430784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 140/252] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:36:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:06.437020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae69b2-6f5a-4f40-975c-c2fddc9234d6 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:06.457344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f6496-a8b9-45c9-8558-c82b56a1246b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:06.497203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980424c-9129-47d8-8c44-7d6605d283f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:06.498181 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980424c-9129-47d8-8c44-7d6605d283f5 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:06.498650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980424c-9129-47d8-8c44-7d6605d283f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:06.499156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980424c-9129-47d8-8c44-7d6605d283f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:06.500022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980424c-9129-47d8-8c44-7d6605d283f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:06.500517 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980424c-9129-47d8-8c44-7d6605d283f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:06.502183 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980424c-9129-47d8-8c44-7d6605d283f5 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:06.513348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 113/253] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:06.518927 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980424c-9129-47d8-8c44-7d6605d283f5 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:06.532918 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:06.588110 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae69b2-6f5a-4f40-975c-c2fddc9234d6 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:06.652295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412560ba-9c9f-40b8-98be-757888ea35d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:06.653748 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412560ba-9c9f-40b8-98be-757888ea35d1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:06.654219 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412560ba-9c9f-40b8-98be-757888ea35d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:06.654704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412560ba-9c9f-40b8-98be-757888ea35d1 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:06.655606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412560ba-9c9f-40b8-98be-757888ea35d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:06.656117 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412560ba-9c9f-40b8-98be-757888ea35d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:06.657705 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412560ba-9c9f-40b8-98be-757888ea35d1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:06.667446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 141/254] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:06.669750 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd32c69-a0be-4d6a-8440-d7e57ba36fc6 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:06.670227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd32c69-a0be-4d6a-8440-d7e57ba36fc6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:06.670695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd32c69-a0be-4d6a-8440-d7e57ba36fc6 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:06.672232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 142/255] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:06.674483 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd32c69-a0be-4d6a-8440-d7e57ba36fc6 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:06.695620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:06.696950 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980424c-9129-47d8-8c44-7d6605d283f5 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:06.739009 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a43780-6745-4e5c-9be2-c402cb68ea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:06.740130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a43780-6745-4e5c-9be2-c402cb68ea1f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:06.740598 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a43780-6745-4e5c-9be2-c402cb68ea1f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:06.741053 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a43780-6745-4e5c-9be2-c402cb68ea1f None admin] PATH_INFO: `/v3/users/476f725ae41944fc923b6419c2edf003` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:06.743363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a43780-6745-4e5c-9be2-c402cb68ea1f None admin] RBAC: Authorizing `identity:get_user(user_id=476f725ae41944fc923b6419c2edf003)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:06.745752 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a43780-6745-4e5c-9be2-c402cb68ea1f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:06.749945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 114/256] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:06 2022] GET /identity/v3/users/476f725ae41944fc923b6419c2edf003 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:06.752700 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f72d2-68d9-4d15-9c55-eebd116623d5 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:06.753560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f72d2-68d9-4d15-9c55-eebd116623d5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:06.755004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f72d2-68d9-4d15-9c55-eebd116623d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:06.794611 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-a99f72d2-68d9-4d15-9c55-eebd116623d5 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:06.797234 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-a99f72d2-68d9-4d15-9c55-eebd116623d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:06.911535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd32c69-a0be-4d6a-8440-d7e57ba36fc6 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:06.913757 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 115/257] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:06.914846 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f72d2-68d9-4d15-9c55-eebd116623d5 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:06.927097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:06.981027 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9e5d3-1615-40d7-a4a1-481201021cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:06.981912 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9e5d3-1615-40d7-a4a1-481201021cd6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:06.982328 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9e5d3-1615-40d7-a4a1-481201021cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:06.983803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9e5d3-1615-40d7-a4a1-481201021cd6 None admin] PATH_INFO: `/v3/users/2b7a90d238674f3ba53b737c52d82da3` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:06.985699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9e5d3-1615-40d7-a4a1-481201021cd6 None admin] RBAC: Authorizing `identity:get_user(user_id=2b7a90d238674f3ba53b737c52d82da3)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:06.987328 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9e5d3-1615-40d7-a4a1-481201021cd6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:06.989283 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 143/258] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:06 2022] GET /identity/v3/users/2b7a90d238674f3ba53b737c52d82da3 => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:06.996729 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9e5d3-1615-40d7-a4a1-481201021cd6 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:07.107017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f72d2-68d9-4d15-9c55-eebd116623d5 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:07.151469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:07.185176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05656a-deb4-4ae6-a442-d609936dbd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:07.185820 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05656a-deb4-4ae6-a442-d609936dbd8a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:07.186089 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05656a-deb4-4ae6-a442-d609936dbd8a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:07.186356 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05656a-deb4-4ae6-a442-d609936dbd8a None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:07.187655 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05656a-deb4-4ae6-a442-d609936dbd8a None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:07.189115 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05656a-deb4-4ae6-a442-d609936dbd8a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:07.190918 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 116/259] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:06 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:07.196714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05656a-deb4-4ae6-a442-d609936dbd8a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:07.205928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9e5d3-1615-40d7-a4a1-481201021cd6 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:07.266544 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190d777-c8fd-4e46-bf1b-7b2a58116a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:07.267185 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190d777-c8fd-4e46-bf1b-7b2a58116a15 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:07.267462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190d777-c8fd-4e46-bf1b-7b2a58116a15 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:07.267729 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190d777-c8fd-4e46-bf1b-7b2a58116a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:07.331487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:07.377779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05656a-deb4-4ae6-a442-d609936dbd8a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:07.397391 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190d777-c8fd-4e46-bf1b-7b2a58116a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:07.398881 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190d777-c8fd-4e46-bf1b-7b2a58116a15 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:07.446901 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c5e10-dec1-4205-b361-4752c200dc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:07.447636 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c5e10-dec1-4205-b361-4752c200dc96 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:07.447866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c5e10-dec1-4205-b361-4752c200dc96 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:07.448131 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c5e10-dec1-4205-b361-4752c200dc96 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144/users/476f725ae41944fc923b6419c2edf003/roles/91e10f3d9d934fe1be9d3642b8092d08` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:07.451228 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c5e10-dec1-4205-b361-4752c200dc96 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7a7423471544bcb8851a5df381c6144, user_id=476f725ae41944fc923b6419c2edf003, role_id=91e10f3d9d934fe1be9d3642b8092d08)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:07.452687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c5e10-dec1-4205-b361-4752c200dc96 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:07.481266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 117/260] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:36:07 2022] PUT /identity/v3/projects/d7a7423471544bcb8851a5df381c6144/users/476f725ae41944fc923b6419c2edf003/roles/91e10f3d9d934fe1be9d3642b8092d08 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:07.482387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c5e10-dec1-4205-b361-4752c200dc96 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:07.529860 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:07.582439 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 144/261] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:07.605450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190d777-c8fd-4e46-bf1b-7b2a58116a15 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:07.687785 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c5e10-dec1-4205-b361-4752c200dc96 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:07.747097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c40aa-e21e-407e-aa4c-3e3e15df10e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:07.747781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c40aa-e21e-407e-aa4c-3e3e15df10e5 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:07.748056 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c40aa-e21e-407e-aa4c-3e3e15df10e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:07.748326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c40aa-e21e-407e-aa4c-3e3e15df10e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:07.750629 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:07.758913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190d777-c8fd-4e46-bf1b-7b2a58116a15 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:07.772828 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c40aa-e21e-407e-aa4c-3e3e15df10e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-684c40aa-e21e-407e-aa4c-3e3e15df10e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:07.778250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:07.779310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:07.780100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:07.782375 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 118/262] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:36:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:07.790385 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c40aa-e21e-407e-aa4c-3e3e15df10e5 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:07.814902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aea15-89ca-4bda-a9b4-4cbef6356a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:07.815579 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aea15-89ca-4bda-a9b4-4cbef6356a3a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:07.815853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aea15-89ca-4bda-a9b4-4cbef6356a3a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:07.816118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aea15-89ca-4bda-a9b4-4cbef6356a3a None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:07.817499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aea15-89ca-4bda-a9b4-4cbef6356a3a None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:07.818963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aea15-89ca-4bda-a9b4-4cbef6356a3a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:07.830669 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 145/263] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:07 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:07.833046 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aea15-89ca-4bda-a9b4-4cbef6356a3a None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:07.927505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:07.990259 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c40aa-e21e-407e-aa4c-3e3e15df10e5 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:08.025771 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f1676-43a7-42a6-9888-0383b7891033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:08.026385 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f1676-43a7-42a6-9888-0383b7891033 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.026661 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f1676-43a7-42a6-9888-0383b7891033 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.026783 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aea15-89ca-4bda-a9b4-4cbef6356a3a None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:08.026961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f1676-43a7-42a6-9888-0383b7891033 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.027626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f1676-43a7-42a6-9888-0383b7891033 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:08.027930 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f1676-43a7-42a6-9888-0383b7891033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:08.029330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f1676-43a7-42a6-9888-0383b7891033 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:08.038531 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 119/264] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:08.044055 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f1676-43a7-42a6-9888-0383b7891033 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:08.104194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:08.105508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bff86-a04d-4f54-985e-e3d864d992e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:08.106131 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bff86-a04d-4f54-985e-e3d864d992e0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.106400 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bff86-a04d-4f54-985e-e3d864d992e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.106660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bff86-a04d-4f54-985e-e3d864d992e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.107512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bff86-a04d-4f54-985e-e3d864d992e0 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=d7a7423471544bcb8851a5df381c6144, user.id=2b7a90d238674f3ba53b737c52d82da3) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:08.107890 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bff86-a04d-4f54-985e-e3d864d992e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:08.110294 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bff86-a04d-4f54-985e-e3d864d992e0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:08.126409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 146/265] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:07 2022] GET /identity/v3/role_assignments?user.id=2b7a90d238674f3ba53b737c52d82da3&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:08.153294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f1676-43a7-42a6-9888-0383b7891033 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:08.190797 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b4eee-9d87-4da7-9da1-3da419841b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:08.191477 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b4eee-9d87-4da7-9da1-3da419841b40 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.191755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b4eee-9d87-4da7-9da1-3da419841b40 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.192027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b4eee-9d87-4da7-9da1-3da419841b40 None admin] PATH_INFO: `/v3/users/2b7a90d238674f3ba53b737c52d82da3` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.193239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b4eee-9d87-4da7-9da1-3da419841b40 None admin] RBAC: Authorizing `identity:get_user(user_id=2b7a90d238674f3ba53b737c52d82da3)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:08.194857 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b4eee-9d87-4da7-9da1-3da419841b40 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:08.196643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 120/266] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:08 2022] GET /identity/v3/users/2b7a90d238674f3ba53b737c52d82da3 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:08.202535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bff86-a04d-4f54-985e-e3d864d992e0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:08.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bff86-a04d-4f54-985e-e3d864d992e0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:08.361232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:08.387611 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65efc1-80e8-49ac-b46a-44ddd382e7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:08.388290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65efc1-80e8-49ac-b46a-44ddd382e7d1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.388560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65efc1-80e8-49ac-b46a-44ddd382e7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.388821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65efc1-80e8-49ac-b46a-44ddd382e7d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.474978 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65efc1-80e8-49ac-b46a-44ddd382e7d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:08.476530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65efc1-80e8-49ac-b46a-44ddd382e7d1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:08.497376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4873b-3c03-4dc3-bbb1-9a89d95bafed None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.497693 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4873b-3c03-4dc3-bbb1-9a89d95bafed None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.497928 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4873b-3c03-4dc3-bbb1-9a89d95bafed None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.499571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 121/267] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:08.508310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e40d6a-d6b5-4032-8ce3-6751f9ef28e8 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.508849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e40d6a-d6b5-4032-8ce3-6751f9ef28e8 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.509318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e40d6a-d6b5-4032-8ce3-6751f9ef28e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.543152 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-07e40d6a-d6b5-4032-8ce3-6751f9ef28e8 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:08.544957 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-07e40d6a-d6b5-4032-8ce3-6751f9ef28e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:08.609732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 147/268] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:08.617321 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65efc1-80e8-49ac-b46a-44ddd382e7d1 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:08.631739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:08.668270 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 122/269] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:08.745831 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65efc1-80e8-49ac-b46a-44ddd382e7d1 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:08.788031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0580ae-a5df-43fd-814e-1ed0d4bf8b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:08.788665 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0580ae-a5df-43fd-814e-1ed0d4bf8b6d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.788921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0580ae-a5df-43fd-814e-1ed0d4bf8b6d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.789244 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0580ae-a5df-43fd-814e-1ed0d4bf8b6d None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.790695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0580ae-a5df-43fd-814e-1ed0d4bf8b6d None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:08.792220 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0580ae-a5df-43fd-814e-1ed0d4bf8b6d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:08.794218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 148/270] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:08 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:08.800097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e40d6a-d6b5-4032-8ce3-6751f9ef28e8 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:08.863365 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:08.915698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e40d6a-d6b5-4032-8ce3-6751f9ef28e8 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:08.940292 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1879fc9-bce6-4d52-bed8-218b1bc83ac5 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.940895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1879fc9-bce6-4d52-bed8-218b1bc83ac5 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.941353 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1879fc9-bce6-4d52-bed8-218b1bc83ac5 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.946762 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 149/271] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:36:08.948227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b2f28-f52d-45b0-b668-f42e89610eaf None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.948734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b2f28-f52d-45b0-b668-f42e89610eaf None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.949186 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b2f28-f52d-45b0-b668-f42e89610eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.968652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-ad0b2f28-f52d-45b0-b668-f42e89610eaf None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:08.970192 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ad0b2f28-f52d-45b0-b668-f42e89610eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:08.974191 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c2dc7-00fe-4890-9b6b-573ee78c5205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:08.974853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c2dc7-00fe-4890-9b6b-573ee78c5205 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:08.975206 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c2dc7-00fe-4890-9b6b-573ee78c5205 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:08.975405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c2dc7-00fe-4890-9b6b-573ee78c5205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:08.976139 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c2dc7-00fe-4890-9b6b-573ee78c5205 None admin] RBAC: Adding query filter params (role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=d7a7423471544bcb8851a5df381c6144, user.id=2b7a90d238674f3ba53b737c52d82da3) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:08.976442 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c2dc7-00fe-4890-9b6b-573ee78c5205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:08.977942 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c2dc7-00fe-4890-9b6b-573ee78c5205 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:08.992210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 123/272] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:08 2022] GET /identity/v3/role_assignments?user.id=2b7a90d238674f3ba53b737c52d82da3&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:09.045679 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 150/273] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:09.103572 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c2dc7-00fe-4890-9b6b-573ee78c5205 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:09.242051 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c2dc7-00fe-4890-9b6b-573ee78c5205 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:09.246365 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b813eb-905b-43fb-8e59-10a3d0520fae None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.246824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b813eb-905b-43fb-8e59-10a3d0520fae None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.247341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b813eb-905b-43fb-8e59-10a3d0520fae None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:09.249087 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 151/274] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:09.254334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba2ec6-c379-428d-82f7-84a0a1cf2c38 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.254616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba2ec6-c379-428d-82f7-84a0a1cf2c38 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.254888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba2ec6-c379-428d-82f7-84a0a1cf2c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:09.262913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:09.270558 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:09.280731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-7aba2ec6-c379-428d-82f7-84a0a1cf2c38 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:09.282658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-7aba2ec6-c379-428d-82f7-84a0a1cf2c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:09.284368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f314d-a8d8-4707-8a29-c33cd5ea75b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:09.285067 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f314d-a8d8-4707-8a29-c33cd5ea75b7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.285531 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f314d-a8d8-4707-8a29-c33cd5ea75b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.285970 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f314d-a8d8-4707-8a29-c33cd5ea75b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:09.298691 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f314d-a8d8-4707-8a29-c33cd5ea75b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-d33f314d-a8d8-4707-8a29-c33cd5ea75b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:09.305473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:09.306353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:09.307265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:09.308875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 124/275] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:09.314675 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f314d-a8d8-4707-8a29-c33cd5ea75b7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:09.378940 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 152/276] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:09.380054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba2ec6-c379-428d-82f7-84a0a1cf2c38 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:09.454117 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f314d-a8d8-4707-8a29-c33cd5ea75b7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:09.490828 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a89af5-7194-48e5-b4e8-f0882d4bd611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:09.491474 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a89af5-7194-48e5-b4e8-f0882d4bd611 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.491749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a89af5-7194-48e5-b4e8-f0882d4bd611 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.492016 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a89af5-7194-48e5-b4e8-f0882d4bd611 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:09.492744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a89af5-7194-48e5-b4e8-f0882d4bd611 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:09.492986 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a89af5-7194-48e5-b4e8-f0882d4bd611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:09.494389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a89af5-7194-48e5-b4e8-f0882d4bd611 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:09.503203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 125/277] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:09.508409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba2ec6-c379-428d-82f7-84a0a1cf2c38 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:09.514837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a89af5-7194-48e5-b4e8-f0882d4bd611 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:09.544745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf1a5-1ba7-4a6f-8515-561ceabe7632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:09.546203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf1a5-1ba7-4a6f-8515-561ceabe7632 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.546666 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf1a5-1ba7-4a6f-8515-561ceabe7632 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.547100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf1a5-1ba7-4a6f-8515-561ceabe7632 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:09.550109 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:09.560111 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bf1a5-1ba7-4a6f-8515-561ceabe7632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:09.561491 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-d56bf1a5-1ba7-4a6f-8515-561ceabe7632 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:09.567884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:09.568875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:09.569656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:09.572235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 153/278] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:09.579220 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf1a5-1ba7-4a6f-8515-561ceabe7632 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:09.637018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a89af5-7194-48e5-b4e8-f0882d4bd611 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:09.681233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eab580-1e9a-4363-ab14-324ce0f4c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:09.681871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eab580-1e9a-4363-ab14-324ce0f4c743 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.682269 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eab580-1e9a-4363-ab14-324ce0f4c743 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.682397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eab580-1e9a-4363-ab14-324ce0f4c743 None admin] PATH_INFO: `/v3/users/a1c7a935c2c84b259030d28b3449b9a3` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:09.683741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eab580-1e9a-4363-ab14-324ce0f4c743 None admin] RBAC: Authorizing `identity:get_user(user_id=a1c7a935c2c84b259030d28b3449b9a3)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:09.685410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eab580-1e9a-4363-ab14-324ce0f4c743 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:09.687170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 126/279] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:09 2022] GET /identity/v3/users/a1c7a935c2c84b259030d28b3449b9a3 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:09.687795 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eab580-1e9a-4363-ab14-324ce0f4c743 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:09.692444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf1a5-1ba7-4a6f-8515-561ceabe7632 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:09.730351 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f5767-9c2b-4c8d-b804-30689bacbfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:09.731423 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f5767-9c2b-4c8d-b804-30689bacbfcb None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.731703 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f5767-9c2b-4c8d-b804-30689bacbfcb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.731970 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f5767-9c2b-4c8d-b804-30689bacbfcb None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:09.733048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f5767-9c2b-4c8d-b804-30689bacbfcb None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:09.733370 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f5767-9c2b-4c8d-b804-30689bacbfcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:09.735423 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f5767-9c2b-4c8d-b804-30689bacbfcb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:09.745972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 154/280] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:09.746558 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f5767-9c2b-4c8d-b804-30689bacbfcb None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:09.798752 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eab580-1e9a-4363-ab14-324ce0f4c743 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:09.808452 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:09.813008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:09.837613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4374155-9c7b-4160-b313-79335023c571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:09.838580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374155-9c7b-4160-b313-79335023c571 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.839032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374155-9c7b-4160-b313-79335023c571 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.839519 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374155-9c7b-4160-b313-79335023c571 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:09.840639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4374155-9c7b-4160-b313-79335023c571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:09.842121 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4374155-9c7b-4160-b313-79335023c571 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:09.848312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 127/281] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:09.849056 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4374155-9c7b-4160-b313-79335023c571 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:09.860745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f5767-9c2b-4c8d-b804-30689bacbfcb None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:09.898389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce6e44-8c90-4cd8-a417-bdf020ee9c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:09.899030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce6e44-8c90-4cd8-a417-bdf020ee9c57 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.899336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce6e44-8c90-4cd8-a417-bdf020ee9c57 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.899603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce6e44-8c90-4cd8-a417-bdf020ee9c57 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:09.901475 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce6e44-8c90-4cd8-a417-bdf020ee9c57 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:09.902942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce6e44-8c90-4cd8-a417-bdf020ee9c57 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:09.909575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 155/282] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:09 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:09.909575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce6e44-8c90-4cd8-a417-bdf020ee9c57 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:09.958753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4374155-9c7b-4160-b313-79335023c571 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:09.991711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a5931-34aa-43cd-9e01-ce8102aaaf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:09.992339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a5931-34aa-43cd-9e01-ce8102aaaf30 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:09.992616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a5931-34aa-43cd-9e01-ce8102aaaf30 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:09.992883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a5931-34aa-43cd-9e01-ce8102aaaf30 None admin] PATH_INFO: `/v3/users/0f680c647dfd4fce99a4fe168eacd2c2` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:10.006042 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a5931-34aa-43cd-9e01-ce8102aaaf30 None admin] RBAC: Authorizing `identity:get_user(user_id=0f680c647dfd4fce99a4fe168eacd2c2)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:10.007526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a5931-34aa-43cd-9e01-ce8102aaaf30 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:10.009191 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 128/283] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:09 2022] GET /identity/v3/users/0f680c647dfd4fce99a4fe168eacd2c2 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:10.009938 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a5931-34aa-43cd-9e01-ce8102aaaf30 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:10.012193 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce6e44-8c90-4cd8-a417-bdf020ee9c57 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:10.041897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:10.046832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:10.052556 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4acaa-4c31-4e60-a9e7-d87fbef546f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:10.053995 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4acaa-4c31-4e60-a9e7-d87fbef546f1 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:10.054325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4acaa-4c31-4e60-a9e7-d87fbef546f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:10.054611 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4acaa-4c31-4e60-a9e7-d87fbef546f1 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:10.055313 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4acaa-4c31-4e60-a9e7-d87fbef546f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:10.057478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4acaa-4c31-4e60-a9e7-d87fbef546f1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:10.078380 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 156/284] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:36:09 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:36:10.079944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235559b-c244-42f8-9704-b4f538c99c16 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:10.080237 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235559b-c244-42f8-9704-b4f538c99c16 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:10.080506 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235559b-c244-42f8-9704-b4f538c99c16 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:10.081768 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 157/285] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:10.082479 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235559b-c244-42f8-9704-b4f538c99c16 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:10.133201 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a5931-34aa-43cd-9e01-ce8102aaaf30 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:10.169314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1104e-3d7e-4df9-891a-f6c108bf8af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:10.169981 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1104e-3d7e-4df9-891a-f6c108bf8af7 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:10.170256 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1104e-3d7e-4df9-891a-f6c108bf8af7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:10.170524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1104e-3d7e-4df9-891a-f6c108bf8af7 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144/users/a1c7a935c2c84b259030d28b3449b9a3/roles/1ece0be4f4c9430b9909bc8d8e87adc3` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:10.172530 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1104e-3d7e-4df9-891a-f6c108bf8af7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7a7423471544bcb8851a5df381c6144, user_id=a1c7a935c2c84b259030d28b3449b9a3, role_id=1ece0be4f4c9430b9909bc8d8e87adc3)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:10.173999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1104e-3d7e-4df9-891a-f6c108bf8af7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:10.188906 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 129/286] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:36:10 2022] PUT /identity/v3/projects/d7a7423471544bcb8851a5df381c6144/users/a1c7a935c2c84b259030d28b3449b9a3/roles/1ece0be4f4c9430b9909bc8d8e87adc3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:10.190798 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8e8f9-7242-4193-8fe6-657f37718008 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:10.191349 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8e8f9-7242-4193-8fe6-657f37718008 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:10.191866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8e8f9-7242-4193-8fe6-657f37718008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:10.211361 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235559b-c244-42f8-9704-b4f538c99c16 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:10.215761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-67c8e8f9-7242-4193-8fe6-657f37718008 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:10.217357 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-67c8e8f9-7242-4193-8fe6-657f37718008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:10.259314 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73554cd0-93d5-41ac-bbcb-ce2c4164e311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:10.259870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73554cd0-93d5-41ac-bbcb-ce2c4164e311 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:10.260147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73554cd0-93d5-41ac-bbcb-ce2c4164e311 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:10.260413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73554cd0-93d5-41ac-bbcb-ce2c4164e311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:10.261149 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73554cd0-93d5-41ac-bbcb-ce2c4164e311 None admin] RBAC: Adding query filter params (role.id=91e10f3d9d934fe1be9d3642b8092d08, user.id=0f680c647dfd4fce99a4fe168eacd2c2, scope.system=all) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:10.261453 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73554cd0-93d5-41ac-bbcb-ce2c4164e311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:10.262948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73554cd0-93d5-41ac-bbcb-ce2c4164e311 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:10.269438 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 158/287] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Aug 30 11:36:10 2022] GET /identity/v3/role_assignments?user.id=0f680c647dfd4fce99a4fe168eacd2c2&scope.system=all&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:10.313525 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 130/288] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:10.580599 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73554cd0-93d5-41ac-bbcb-ce2c4164e311 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:10.636064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:10.710154 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73554cd0-93d5-41ac-bbcb-ce2c4164e311 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:10.756804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf34517-f8fb-406d-82df-be717e88c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:10.757831 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf34517-f8fb-406d-82df-be717e88c20f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:10.758299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf34517-f8fb-406d-82df-be717e88c20f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:10.758786 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf34517-f8fb-406d-82df-be717e88c20f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:10.780037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf34517-f8fb-406d-82df-be717e88c20f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-1cf34517-f8fb-406d-82df-be717e88c20f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:10.785751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:10.786604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:10.787559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:10.790098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 159/289] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:10.795875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8e8f9-7242-4193-8fe6-657f37718008 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:10.835918 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:10.937008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8e8f9-7242-4193-8fe6-657f37718008 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:10.978641 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb98fc1-6d0d-4298-81ef-d8241be8c534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:10.979282 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb98fc1-6d0d-4298-81ef-d8241be8c534 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:10.979555 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb98fc1-6d0d-4298-81ef-d8241be8c534 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:10.979820 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb98fc1-6d0d-4298-81ef-d8241be8c534 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:10.980500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb98fc1-6d0d-4298-81ef-d8241be8c534 None admin] RBAC: Adding query filter params (name=member) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:10.980798 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb98fc1-6d0d-4298-81ef-d8241be8c534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:10.982188 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb98fc1-6d0d-4298-81ef-d8241be8c534 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:10.994820 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 131/290] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:11.000394 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf34517-f8fb-406d-82df-be717e88c20f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:11.150714 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf34517-f8fb-406d-82df-be717e88c20f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:11.161818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:11.198046 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a49164-1ec9-4723-8e31-2ef04a053c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:11.198685 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a49164-1ec9-4723-8e31-2ef04a053c45 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:11.198959 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a49164-1ec9-4723-8e31-2ef04a053c45 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:11.199264 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a49164-1ec9-4723-8e31-2ef04a053c45 None admin] PATH_INFO: `/v3/users/476f725ae41944fc923b6419c2edf003` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:11.200395 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a49164-1ec9-4723-8e31-2ef04a053c45 None admin] RBAC: Authorizing `identity:get_user(user_id=476f725ae41944fc923b6419c2edf003)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:11.201861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a49164-1ec9-4723-8e31-2ef04a053c45 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:11.204095 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 160/291] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:10 2022] GET /identity/v3/users/476f725ae41944fc923b6419c2edf003 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:11.209777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb98fc1-6d0d-4298-81ef-d8241be8c534 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:11.305381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:11.319876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb98fc1-6d0d-4298-81ef-d8241be8c534 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:11.364079 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6bc83-1b3a-42af-b75e-48a7b58a950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:11.364712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6bc83-1b3a-42af-b75e-48a7b58a950f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:11.365057 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6bc83-1b3a-42af-b75e-48a7b58a950f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:11.365327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6bc83-1b3a-42af-b75e-48a7b58a950f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:11.453413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6bc83-1b3a-42af-b75e-48a7b58a950f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:11.455230 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6bc83-1b3a-42af-b75e-48a7b58a950f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:11.538039 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:11.558627 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 132/292] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:11.564210 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a49164-1ec9-4723-8e31-2ef04a053c45 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:11.679890 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a49164-1ec9-4723-8e31-2ef04a053c45 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:11.727325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01211d48-0ad9-4058-b815-c2f7066add79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:11.728347 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211d48-0ad9-4058-b815-c2f7066add79 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:11.728841 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211d48-0ad9-4058-b815-c2f7066add79 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:11.729308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211d48-0ad9-4058-b815-c2f7066add79 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:11.736243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01211d48-0ad9-4058-b815-c2f7066add79 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:11.738903 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01211d48-0ad9-4058-b815-c2f7066add79 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:11.741893 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 161/293] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:11 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:11.747785 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6bc83-1b3a-42af-b75e-48a7b58a950f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:11.878928 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6bc83-1b3a-42af-b75e-48a7b58a950f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:11.922325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad283ed-a67c-41ef-9314-0e1b556a4d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:11.922964 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad283ed-a67c-41ef-9314-0e1b556a4d99 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:11.923271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad283ed-a67c-41ef-9314-0e1b556a4d99 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:11.923537 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad283ed-a67c-41ef-9314-0e1b556a4d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:11.924292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad283ed-a67c-41ef-9314-0e1b556a4d99 None admin] RBAC: Adding query filter params (role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=d7a7423471544bcb8851a5df381c6144, user.id=476f725ae41944fc923b6419c2edf003) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:11.924597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad283ed-a67c-41ef-9314-0e1b556a4d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:11.926094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad283ed-a67c-41ef-9314-0e1b556a4d99 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:11.932854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 133/294] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:11 2022] GET /identity/v3/role_assignments?user.id=476f725ae41944fc923b6419c2edf003&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:12.485478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dd437-2565-49cd-afc6-09ab03e78063 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:12.485981 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dd437-2565-49cd-afc6-09ab03e78063 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:12.486055 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dd437-2565-49cd-afc6-09ab03e78063 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:12.487463 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 162/295] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:12.492956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9587-7a6c-470c-9286-cc93ee6cdf39 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:12.493235 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9587-7a6c-470c-9286-cc93ee6cdf39 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:12.493502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9587-7a6c-470c-9286-cc93ee6cdf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:12.514851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-0bdf9587-7a6c-470c-9286-cc93ee6cdf39 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:12.516384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-0bdf9587-7a6c-470c-9286-cc93ee6cdf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:12.536084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d117ad-6607-4a43-8f45-8af51045b312 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:12.536084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d117ad-6607-4a43-8f45-8af51045b312 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:12.536328 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d117ad-6607-4a43-8f45-8af51045b312 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:12.537926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 163/296] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:12.545518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1cfce-2489-4557-af93-16ad3580d3e3 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:12.546104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1cfce-2489-4557-af93-16ad3580d3e3 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:12.546374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1cfce-2489-4557-af93-16ad3580d3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:12.569687 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-59d1cfce-2489-4557-af93-16ad3580d3e3 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:12.570946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-59d1cfce-2489-4557-af93-16ad3580d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:12.573144 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:12.595158 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 134/297] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:12.596312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67893c5-a86e-45ce-a9ea-42139461b99e None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:12.596592 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67893c5-a86e-45ce-a9ea-42139461b99e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:12.596859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67893c5-a86e-45ce-a9ea-42139461b99e None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:12.598104 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 135/298] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:12.605214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367200f-420a-4d64-b4eb-9321f0554a0e None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:12.605562 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367200f-420a-4d64-b4eb-9321f0554a0e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:12.605899 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367200f-420a-4d64-b4eb-9321f0554a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:12.620634 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:12.630761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-8367200f-420a-4d64-b4eb-9321f0554a0e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:12.632033 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-8367200f-420a-4d64-b4eb-9321f0554a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:12.663193 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 164/299] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:12.701672 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 136/300] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:12.777525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41570f36-bf82-4991-b160-cd78b8ccb22a None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:12.778150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41570f36-bf82-4991-b160-cd78b8ccb22a None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:12.778613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41570f36-bf82-4991-b160-cd78b8ccb22a None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:12.780471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 165/301] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:12.785927 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e79bb-2765-41db-8bde-3fc370f0c128 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:12.786211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e79bb-2765-41db-8bde-3fc370f0c128 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:12.786482 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e79bb-2765-41db-8bde-3fc370f0c128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:12.808724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-221e79bb-2765-41db-8bde-3fc370f0c128 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:12.810174 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-221e79bb-2765-41db-8bde-3fc370f0c128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:12.877024 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 137/302] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:12.898311 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41570f36-bf82-4991-b160-cd78b8ccb22a None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:12.925491 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e79bb-2765-41db-8bde-3fc370f0c128 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:12.979371 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:13.011889 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41570f36-bf82-4991-b160-cd78b8ccb22a None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.041213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e79bb-2765-41db-8bde-3fc370f0c128 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.046256 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec085a1-da34-4dab-99d9-130609564d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.046941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec085a1-da34-4dab-99d9-130609564d8f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.047287 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec085a1-da34-4dab-99d9-130609564d8f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.047553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec085a1-da34-4dab-99d9-130609564d8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.058525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec085a1-da34-4dab-99d9-130609564d8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-cec085a1-da34-4dab-99d9-130609564d8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:13.063561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:13.064444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:13.065231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 166/303] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:13.065872 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec085a1-da34-4dab-99d9-130609564d8f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.080207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49424169-72b1-4e6c-81a5-cd427f3c7f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.080925 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49424169-72b1-4e6c-81a5-cd427f3c7f55 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.081203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49424169-72b1-4e6c-81a5-cd427f3c7f55 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.081477 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49424169-72b1-4e6c-81a5-cd427f3c7f55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.092652 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49424169-72b1-4e6c-81a5-cd427f3c7f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-49424169-72b1-4e6c-81a5-cd427f3c7f55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:13.097524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:13.098376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:13.099252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 138/304] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:13.099986 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49424169-72b1-4e6c-81a5-cd427f3c7f55 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.104194 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:13.175614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec085a1-da34-4dab-99d9-130609564d8f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.204873 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:13.208039 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49424169-72b1-4e6c-81a5-cd427f3c7f55 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.208498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ec3ba-cbea-4226-bc33-fd17b155f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.209133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ec3ba-cbea-4226-bc33-fd17b155f530 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.209405 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ec3ba-cbea-4226-bc33-fd17b155f530 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.209668 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ec3ba-cbea-4226-bc33-fd17b155f530 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.219548 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ec3ba-cbea-4226-bc33-fd17b155f530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-d34ec3ba-cbea-4226-bc33-fd17b155f530 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:13.224431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:13.225524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 167/305] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:13.226514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ec3ba-cbea-4226-bc33-fd17b155f530 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.246087 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd2ce8-ae9b-4b92-b333-ae2224f0c8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.246622 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd2ce8-ae9b-4b92-b333-ae2224f0c8e8 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.246894 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd2ce8-ae9b-4b92-b333-ae2224f0c8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.247181 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd2ce8-ae9b-4b92-b333-ae2224f0c8e8 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.247842 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd2ce8-ae9b-4b92-b333-ae2224f0c8e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:13.248176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd2ce8-ae9b-4b92-b333-ae2224f0c8e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:13.249557 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd2ce8-ae9b-4b92-b333-ae2224f0c8e8 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:13.257794 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 139/306] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:13.258559 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd2ce8-ae9b-4b92-b333-ae2224f0c8e8 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.317189 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:13.326804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ec3ba-cbea-4226-bc33-fd17b155f530 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.355050 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd2ce8-ae9b-4b92-b333-ae2224f0c8e8 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.362385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451f23d-ec9a-4b0a-9085-3894b0f3caf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.362999 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451f23d-ec9a-4b0a-9085-3894b0f3caf3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.363308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451f23d-ec9a-4b0a-9085-3894b0f3caf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.363580 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451f23d-ec9a-4b0a-9085-3894b0f3caf3 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.364242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451f23d-ec9a-4b0a-9085-3894b0f3caf3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:13.364540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451f23d-ec9a-4b0a-9085-3894b0f3caf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:13.365926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451f23d-ec9a-4b0a-9085-3894b0f3caf3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:13.373888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 168/307] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:13.374666 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451f23d-ec9a-4b0a-9085-3894b0f3caf3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.389908 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265516b-d450-4449-8950-5f61a0b019ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.390538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265516b-d450-4449-8950-5f61a0b019ec None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.390807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265516b-d450-4449-8950-5f61a0b019ec None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.391068 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265516b-d450-4449-8950-5f61a0b019ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.392295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265516b-d450-4449-8950-5f61a0b019ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:13.393776 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265516b-d450-4449-8950-5f61a0b019ec None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:13.395373 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 140/308] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:13.396148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265516b-d450-4449-8950-5f61a0b019ec None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.428761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:13.474414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451f23d-ec9a-4b0a-9085-3894b0f3caf3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.497413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265516b-d450-4449-8950-5f61a0b019ec None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.508579 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78eece-b0e7-4927-ad87-685703d0344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.509192 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78eece-b0e7-4927-ad87-685703d0344c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.509468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78eece-b0e7-4927-ad87-685703d0344c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.509738 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78eece-b0e7-4927-ad87-685703d0344c None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.510446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78eece-b0e7-4927-ad87-685703d0344c None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:13.510723 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78eece-b0e7-4927-ad87-685703d0344c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:13.512141 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78eece-b0e7-4927-ad87-685703d0344c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:13.520137 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 169/309] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:13.520905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78eece-b0e7-4927-ad87-685703d0344c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.530038 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7bcce-e5f6-4c99-bb86-aa2967a70388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.530667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7bcce-e5f6-4c99-bb86-aa2967a70388 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.530939 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7bcce-e5f6-4c99-bb86-aa2967a70388 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.531239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7bcce-e5f6-4c99-bb86-aa2967a70388 None admin] PATH_INFO: `/v3/users/f24df2764f884a9ea42f0deddea2de42` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.538494 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:13.545738 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7bcce-e5f6-4c99-bb86-aa2967a70388 None admin] RBAC: Authorizing `identity:get_user(user_id=f24df2764f884a9ea42f0deddea2de42)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:13.547298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7bcce-e5f6-4c99-bb86-aa2967a70388 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:13.548953 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 141/310] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/users/f24df2764f884a9ea42f0deddea2de42 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:13.549698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7bcce-e5f6-4c99-bb86-aa2967a70388 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.620404 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78eece-b0e7-4927-ad87-685703d0344c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.634761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:13.647208 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7bcce-e5f6-4c99-bb86-aa2967a70388 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.653835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266a00f-75f5-4e35-9085-9079fa2dce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.654523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266a00f-75f5-4e35-9085-9079fa2dce20 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.654799 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266a00f-75f5-4e35-9085-9079fa2dce20 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.655062 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266a00f-75f5-4e35-9085-9079fa2dce20 None admin] PATH_INFO: `/v3/users/a1c7a935c2c84b259030d28b3449b9a3` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.656216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266a00f-75f5-4e35-9085-9079fa2dce20 None admin] RBAC: Authorizing `identity:get_user(user_id=a1c7a935c2c84b259030d28b3449b9a3)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:13.657664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266a00f-75f5-4e35-9085-9079fa2dce20 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:13.659276 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 170/311] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/users/a1c7a935c2c84b259030d28b3449b9a3 => generated 297 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:13.660022 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266a00f-75f5-4e35-9085-9079fa2dce20 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.679782 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38603164-d693-4ef8-8d6f-ac7004c0649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.680387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38603164-d693-4ef8-8d6f-ac7004c0649d None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.680658 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38603164-d693-4ef8-8d6f-ac7004c0649d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.680923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38603164-d693-4ef8-8d6f-ac7004c0649d None admin] PATH_INFO: `/v3/groups` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.681547 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38603164-d693-4ef8-8d6f-ac7004c0649d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:13.682970 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38603164-d693-4ef8-8d6f-ac7004c0649d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:13.699092 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 142/312] 10.222.0.16 () {60 vars in 1134 bytes} [Tue Aug 30 11:36:13 2022] POST /identity/v3/groups => generated 229 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:36:13.700240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38603164-d693-4ef8-8d6f-ac7004c0649d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.760346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:13.765546 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266a00f-75f5-4e35-9085-9079fa2dce20 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.804106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fd4c3-ec19-4269-a229-b44adaa7a6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.804760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fd4c3-ec19-4269-a229-b44adaa7a6ba None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.805036 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fd4c3-ec19-4269-a229-b44adaa7a6ba None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.805300 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fd4c3-ec19-4269-a229-b44adaa7a6ba None admin] PATH_INFO: `/v3/users/0f680c647dfd4fce99a4fe168eacd2c2` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.806490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fd4c3-ec19-4269-a229-b44adaa7a6ba None admin] RBAC: Authorizing `identity:get_user(user_id=0f680c647dfd4fce99a4fe168eacd2c2)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:13.808105 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fd4c3-ec19-4269-a229-b44adaa7a6ba None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:13.809699 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38603164-d693-4ef8-8d6f-ac7004c0649d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.809812 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 171/313] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/users/0f680c647dfd4fce99a4fe168eacd2c2 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:13.815668 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fd4c3-ec19-4269-a229-b44adaa7a6ba None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.849917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590177f-14c5-4f07-a0bb-f0dffdd31085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:13.850511 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590177f-14c5-4f07-a0bb-f0dffdd31085 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:13.850780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590177f-14c5-4f07-a0bb-f0dffdd31085 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:13.851049 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590177f-14c5-4f07-a0bb-f0dffdd31085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:13.851810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590177f-14c5-4f07-a0bb-f0dffdd31085 None admin] RBAC: Adding query filter params (role.id=1ece0be4f4c9430b9909bc8d8e87adc3, user.id=f24df2764f884a9ea42f0deddea2de42, scope.system=all) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:13.852111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590177f-14c5-4f07-a0bb-f0dffdd31085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:13.854360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590177f-14c5-4f07-a0bb-f0dffdd31085 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:13.860157 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 143/314] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/role_assignments?user.id=f24df2764f884a9ea42f0deddea2de42&scope.system=all&role.id=1ece0be4f4c9430b9909bc8d8e87adc3 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:13.861004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590177f-14c5-4f07-a0bb-f0dffdd31085 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:13.879226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:13.954735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fd4c3-ec19-4269-a229-b44adaa7a6ba None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:13.985073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590177f-14c5-4f07-a0bb-f0dffdd31085 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:14.004432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df833aee-6e8e-46f4-aa4c-e799b7f01fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:14.005978 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df833aee-6e8e-46f4-aa4c-e799b7f01fa1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:14.006452 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df833aee-6e8e-46f4-aa4c-e799b7f01fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:14.006984 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df833aee-6e8e-46f4-aa4c-e799b7f01fa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:14.024044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc701a-fced-4339-9149-43a593fcf245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:14.024801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc701a-fced-4339-9149-43a593fcf245 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:14.026132 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc701a-fced-4339-9149-43a593fcf245 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:14.026132 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc701a-fced-4339-9149-43a593fcf245 None admin] PATH_INFO: `/v3/system/users/0f680c647dfd4fce99a4fe168eacd2c2/roles/91e10f3d9d934fe1be9d3642b8092d08` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:14.026804 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc701a-fced-4339-9149-43a593fcf245 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0f680c647dfd4fce99a4fe168eacd2c2, role_id=91e10f3d9d934fe1be9d3642b8092d08)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:14.028223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc701a-fced-4339-9149-43a593fcf245 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:14.035681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:14.044091 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 144/315] 10.222.0.16 () {58 vars in 1334 bytes} [Tue Aug 30 11:36:13 2022] PUT /identity/v3/system/users/0f680c647dfd4fce99a4fe168eacd2c2/roles/91e10f3d9d934fe1be9d3642b8092d08 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:14.097204 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df833aee-6e8e-46f4-aa4c-e799b7f01fa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:14.098732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df833aee-6e8e-46f4-aa4c-e799b7f01fa1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:14.213263 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 172/316] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:14.218881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc701a-fced-4339-9149-43a593fcf245 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:14.299810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:14.324618 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc701a-fced-4339-9149-43a593fcf245 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:14.359681 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eb72e-3d9f-49d3-9ef9-8bf7e8f8bb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:14.360313 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eb72e-3d9f-49d3-9ef9-8bf7e8f8bb8f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:14.360587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eb72e-3d9f-49d3-9ef9-8bf7e8f8bb8f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:14.360855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eb72e-3d9f-49d3-9ef9-8bf7e8f8bb8f None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:14.361974 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eb72e-3d9f-49d3-9ef9-8bf7e8f8bb8f None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:14.363470 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eb72e-3d9f-49d3-9ef9-8bf7e8f8bb8f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:14.365367 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 145/317] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:14 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:14.371196 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df833aee-6e8e-46f4-aa4c-e799b7f01fa1 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:14.427173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:14.476939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df833aee-6e8e-46f4-aa4c-e799b7f01fa1 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:14.511503 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be1475-e8ee-45d8-a539-92724409f0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:14.512140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be1475-e8ee-45d8-a539-92724409f0df None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:14.512376 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be1475-e8ee-45d8-a539-92724409f0df None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:14.512645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be1475-e8ee-45d8-a539-92724409f0df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:14.513371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be1475-e8ee-45d8-a539-92724409f0df None admin] RBAC: Adding query filter params (role.id=1ece0be4f4c9430b9909bc8d8e87adc3, scope.project.id=d7a7423471544bcb8851a5df381c6144, user.id=a1c7a935c2c84b259030d28b3449b9a3) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:14.513671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be1475-e8ee-45d8-a539-92724409f0df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:14.515299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be1475-e8ee-45d8-a539-92724409f0df None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:14.527219 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 173/318] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:14 2022] GET /identity/v3/role_assignments?user.id=a1c7a935c2c84b259030d28b3449b9a3&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=1ece0be4f4c9430b9909bc8d8e87adc3 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:15.674461 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbce415-6a99-42bb-abb0-7e23a026bdd5 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:15.675004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbce415-6a99-42bb-abb0-7e23a026bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:15.675004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbce415-6a99-42bb-abb0-7e23a026bdd5 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:15.676419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 146/319] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:15.681751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e25ee5-563a-4b01-ba55-7e7722db28cb None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:15.682180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e25ee5-563a-4b01-ba55-7e7722db28cb None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:15.682274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e25ee5-563a-4b01-ba55-7e7722db28cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:15.701234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-f5e25ee5-563a-4b01-ba55-7e7722db28cb None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:15.702325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f5e25ee5-563a-4b01-ba55-7e7722db28cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:15.772550 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 174/320] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:15.794622 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b007f-a6a5-4a91-9d24-0681a6f1151a None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:15.794994 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b007f-a6a5-4a91-9d24-0681a6f1151a None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:15.795374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b007f-a6a5-4a91-9d24-0681a6f1151a None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:15.799158 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 147/321] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:15.802088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa05994-dd14-47a4-82fb-087c376e24c2 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:15.802456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa05994-dd14-47a4-82fb-087c376e24c2 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:15.802729 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa05994-dd14-47a4-82fb-087c376e24c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:15.821334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-4aa05994-dd14-47a4-82fb-087c376e24c2 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:15.822514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-4aa05994-dd14-47a4-82fb-087c376e24c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:15.874933 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:15.901437 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 175/322] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:15.986484 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307459e-abd1-40aa-93f0-072746f7ad4f None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:15.986802 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307459e-abd1-40aa-93f0-072746f7ad4f None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:15.986989 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307459e-abd1-40aa-93f0-072746f7ad4f None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:15.988561 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 148/323] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:15.993595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259cf4e-5d1f-45a1-bcea-0ce01a24f435 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:15.993870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259cf4e-5d1f-45a1-bcea-0ce01a24f435 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:15.994094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259cf4e-5d1f-45a1-bcea-0ce01a24f435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.012694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-c259cf4e-5d1f-45a1-bcea-0ce01a24f435 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:16.013857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c259cf4e-5d1f-45a1-bcea-0ce01a24f435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:16.038293 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307459e-abd1-40aa-93f0-072746f7ad4f None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:16.084892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 176/324] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:16.143855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307459e-abd1-40aa-93f0-072746f7ad4f None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:16.163242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259cf4e-5d1f-45a1-bcea-0ce01a24f435 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:16.177647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85850916-6606-4c85-a536-3dff22e24691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:16.178283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850916-6606-4c85-a536-3dff22e24691 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:16.178559 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850916-6606-4c85-a536-3dff22e24691 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:16.178847 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850916-6606-4c85-a536-3dff22e24691 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.179967 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85850916-6606-4c85-a536-3dff22e24691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:16.181412 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85850916-6606-4c85-a536-3dff22e24691 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:16.182988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 149/325] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:16.188280 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85850916-6606-4c85-a536-3dff22e24691 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:16.203309 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:16.288215 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85850916-6606-4c85-a536-3dff22e24691 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:16.288741 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259cf4e-5d1f-45a1-bcea-0ce01a24f435 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:16.297939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:16.326708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43268d-281f-4f45-b9d7-96aa3307897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:16.327457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43268d-281f-4f45-b9d7-96aa3307897c None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:16.327895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43268d-281f-4f45-b9d7-96aa3307897c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:16.328169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43268d-281f-4f45-b9d7-96aa3307897c None admin] PATH_INFO: `/v3/groups` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.328414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc321c2b-efc1-469b-a7d0-dfde70a348f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:16.329115 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43268d-281f-4f45-b9d7-96aa3307897c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:16.329365 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc321c2b-efc1-469b-a7d0-dfde70a348f9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:16.329690 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc321c2b-efc1-469b-a7d0-dfde70a348f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:16.329961 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc321c2b-efc1-469b-a7d0-dfde70a348f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.330608 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43268d-281f-4f45-b9d7-96aa3307897c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:16.341624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc321c2b-efc1-469b-a7d0-dfde70a348f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:16.355459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 150/326] 10.222.0.16 () {60 vars in 1134 bytes} [Tue Aug 30 11:36:16 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-bc321c2b-efc1-469b-a7d0-dfde70a348f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:16.355694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:16.356489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:16.357299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 177/327] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:16.357987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43268d-281f-4f45-b9d7-96aa3307897c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:16.358067 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc321c2b-efc1-469b-a7d0-dfde70a348f9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:16.440242 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:16.479108 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc321c2b-efc1-469b-a7d0-dfde70a348f9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:16.483149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43268d-281f-4f45-b9d7-96aa3307897c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:16.518878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc94b5-7246-49a8-8a86-4f9c71b78131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:16.519721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc94b5-7246-49a8-8a86-4f9c71b78131 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:16.520054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc94b5-7246-49a8-8a86-4f9c71b78131 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:16.520324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc94b5-7246-49a8-8a86-4f9c71b78131 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.521104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc94b5-7246-49a8-8a86-4f9c71b78131 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:16.521404 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc94b5-7246-49a8-8a86-4f9c71b78131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:16.522492 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b564660-ba98-4766-8278-3507cd9879b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:16.522800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc94b5-7246-49a8-8a86-4f9c71b78131 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:16.523043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b564660-ba98-4766-8278-3507cd9879b6 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:16.523428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b564660-ba98-4766-8278-3507cd9879b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:16.523699 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b564660-ba98-4766-8278-3507cd9879b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.532242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 178/328] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:16.537834 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc94b5-7246-49a8-8a86-4f9c71b78131 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:16.538692 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b564660-ba98-4766-8278-3507cd9879b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-7b564660-ba98-4766-8278-3507cd9879b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:16.545283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:16.546360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:16.547364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 151/329] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:16.552111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b564660-ba98-4766-8278-3507cd9879b6 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:16.573626 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:16.668937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc94b5-7246-49a8-8a86-4f9c71b78131 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:16.677610 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b564660-ba98-4766-8278-3507cd9879b6 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:16.714718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:16.723554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425a4ef-1b1b-4d57-9bb6-077bfcb2bdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:16.724714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeac176-c8ac-4f15-827d-b02f4495efef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:16.725480 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeac176-c8ac-4f15-827d-b02f4495efef None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:16.726258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425a4ef-1b1b-4d57-9bb6-077bfcb2bdf2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:16.726527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425a4ef-1b1b-4d57-9bb6-077bfcb2bdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:16.726782 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425a4ef-1b1b-4d57-9bb6-077bfcb2bdf2 None admin] PATH_INFO: `/v3/users/f24df2764f884a9ea42f0deddea2de42` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.727556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeac176-c8ac-4f15-827d-b02f4495efef None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:16.727911 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeac176-c8ac-4f15-827d-b02f4495efef None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.729232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeac176-c8ac-4f15-827d-b02f4495efef None admin] RBAC: Adding query filter params (name=member) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:16.729232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeac176-c8ac-4f15-827d-b02f4495efef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:16.730145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425a4ef-1b1b-4d57-9bb6-077bfcb2bdf2 None admin] RBAC: Authorizing `identity:get_user(user_id=f24df2764f884a9ea42f0deddea2de42)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:16.730590 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeac176-c8ac-4f15-827d-b02f4495efef None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:16.732927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425a4ef-1b1b-4d57-9bb6-077bfcb2bdf2 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:16.735155 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 179/330] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:16 2022] GET /identity/v3/users/f24df2764f884a9ea42f0deddea2de42 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:16.743145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 152/331] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:16.744119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425a4ef-1b1b-4d57-9bb6-077bfcb2bdf2 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:16.747667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeac176-c8ac-4f15-827d-b02f4495efef None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:16.867860 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:16.874690 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425a4ef-1b1b-4d57-9bb6-077bfcb2bdf2 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:16.886557 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeac176-c8ac-4f15-827d-b02f4495efef None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:16.912886 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b339a73-f9a4-4f17-a3f2-5d283f8db803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:16.914125 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b339a73-f9a4-4f17-a3f2-5d283f8db803 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:16.914596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b339a73-f9a4-4f17-a3f2-5d283f8db803 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:16.915187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b339a73-f9a4-4f17-a3f2-5d283f8db803 None admin] PATH_INFO: `/v3/system/users/f24df2764f884a9ea42f0deddea2de42/roles/1ece0be4f4c9430b9909bc8d8e87adc3` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.917556 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b339a73-f9a4-4f17-a3f2-5d283f8db803 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f24df2764f884a9ea42f0deddea2de42, role_id=1ece0be4f4c9430b9909bc8d8e87adc3)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:16.919306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b339a73-f9a4-4f17-a3f2-5d283f8db803 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:16.932236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 180/332] 10.222.0.16 () {58 vars in 1334 bytes} [Tue Aug 30 11:36:16 2022] PUT /identity/v3/system/users/f24df2764f884a9ea42f0deddea2de42/roles/1ece0be4f4c9430b9909bc8d8e87adc3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:16.947642 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afc45d-e3ae-4aba-bdfc-152c2f55dcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:16.948803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afc45d-e3ae-4aba-bdfc-152c2f55dcd3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:16.949423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afc45d-e3ae-4aba-bdfc-152c2f55dcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:16.950092 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afc45d-e3ae-4aba-bdfc-152c2f55dcd3 None admin] PATH_INFO: `/v3/users/0f680c647dfd4fce99a4fe168eacd2c2` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:16.952695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afc45d-e3ae-4aba-bdfc-152c2f55dcd3 None admin] RBAC: Authorizing `identity:get_user(user_id=0f680c647dfd4fce99a4fe168eacd2c2)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:16.956267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afc45d-e3ae-4aba-bdfc-152c2f55dcd3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:16.959098 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 153/333] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:16 2022] GET /identity/v3/users/0f680c647dfd4fce99a4fe168eacd2c2 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:16.965505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b339a73-f9a4-4f17-a3f2-5d283f8db803 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:17.075493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b339a73-f9a4-4f17-a3f2-5d283f8db803 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:17.112406 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f810-bf71-44d1-9073-4faf09af99a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:17.113335 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f810-bf71-44d1-9073-4faf09af99a7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:17.113781 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f810-bf71-44d1-9073-4faf09af99a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:17.114351 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f810-bf71-44d1-9073-4faf09af99a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:17.115353 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f810-bf71-44d1-9073-4faf09af99a7 None admin] RBAC: Adding query filter params (role.id=91e10f3d9d934fe1be9d3642b8092d08, user.id=0f680c647dfd4fce99a4fe168eacd2c2, scope.system=all) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:17.115830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f810-bf71-44d1-9073-4faf09af99a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:17.117489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f810-bf71-44d1-9073-4faf09af99a7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:17.131603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 181/334] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Aug 30 11:36:16 2022] GET /identity/v3/role_assignments?user.id=0f680c647dfd4fce99a4fe168eacd2c2&scope.system=all&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:18.515342 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540328e5-11d5-4738-91c2-6975c5b9cccd None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:18.515841 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540328e5-11d5-4738-91c2-6975c5b9cccd None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:18.515914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540328e5-11d5-4738-91c2-6975c5b9cccd None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:18.517258 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 154/335] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:18.522757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45938db2-b3ae-4913-9078-0c4cf8aebdaf None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:18.523232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45938db2-b3ae-4913-9078-0c4cf8aebdaf None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:18.523407 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45938db2-b3ae-4913-9078-0c4cf8aebdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:18.548596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-45938db2-b3ae-4913-9078-0c4cf8aebdaf None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:18.549782 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-45938db2-b3ae-4913-9078-0c4cf8aebdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:18.566229 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:18.638790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 182/336] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:18.656266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd66cf7-2292-48fd-9787-0f545d533fb5 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:18.656582 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd66cf7-2292-48fd-9787-0f545d533fb5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:18.656809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd66cf7-2292-48fd-9787-0f545d533fb5 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:18.658394 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 155/337] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:18.665258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115103f6-34ba-4ef6-be6d-25f54a5a3999 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:18.665773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115103f6-34ba-4ef6-be6d-25f54a5a3999 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:18.666229 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115103f6-34ba-4ef6-be6d-25f54a5a3999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:18.686730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-115103f6-34ba-4ef6-be6d-25f54a5a3999 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:18.688249 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-115103f6-34ba-4ef6-be6d-25f54a5a3999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:18.761931 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 183/338] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:18.966351 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd66cf7-2292-48fd-9787-0f545d533fb5 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:19.043934 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:19.093204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd66cf7-2292-48fd-9787-0f545d533fb5 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:19.136447 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182cc81-c0cb-4914-8513-0548050f8bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:19.137080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182cc81-c0cb-4914-8513-0548050f8bd6 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.137351 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182cc81-c0cb-4914-8513-0548050f8bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.137608 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182cc81-c0cb-4914-8513-0548050f8bd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.155993 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182cc81-c0cb-4914-8513-0548050f8bd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-4182cc81-c0cb-4914-8513-0548050f8bd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:19.160883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:19.161819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:19.162616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 156/339] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:19.168184 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115103f6-34ba-4ef6-be6d-25f54a5a3999 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:19.258607 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:19.262598 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182cc81-c0cb-4914-8513-0548050f8bd6 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:19.286231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115103f6-34ba-4ef6-be6d-25f54a5a3999 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:19.331874 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b3c83-470a-437d-bd44-f9246dadfdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:19.336469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b3c83-470a-437d-bd44-f9246dadfdb0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.336469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b3c83-470a-437d-bd44-f9246dadfdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.336469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b3c83-470a-437d-bd44-f9246dadfdb0 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.336469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b3c83-470a-437d-bd44-f9246dadfdb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:19.336469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b3c83-470a-437d-bd44-f9246dadfdb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:19.336469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b3c83-470a-437d-bd44-f9246dadfdb0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:19.344305 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 184/340] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:19.349910 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b3c83-470a-437d-bd44-f9246dadfdb0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:19.400709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182cc81-c0cb-4914-8513-0548050f8bd6 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:19.449976 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:19.464825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b3c83-470a-437d-bd44-f9246dadfdb0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:19.498511 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d971c-c915-4060-b5fe-efdbe88a62cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:19.499186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d971c-c915-4060-b5fe-efdbe88a62cf None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.499467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d971c-c915-4060-b5fe-efdbe88a62cf None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.499737 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d971c-c915-4060-b5fe-efdbe88a62cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.503643 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f24ab1-f527-4d8d-8ab8-0151b24abbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:19.504268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f24ab1-f527-4d8d-8ab8-0151b24abbc3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.504541 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f24ab1-f527-4d8d-8ab8-0151b24abbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.504804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f24ab1-f527-4d8d-8ab8-0151b24abbc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.524570 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:19.533099 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d971c-c915-4060-b5fe-efdbe88a62cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-a56d971c-c915-4060-b5fe-efdbe88a62cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:19.538226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:19.539080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:19.539941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:19.547594 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 157/341] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:36:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:19.548977 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112254eb-d87e-44d7-8448-646e2597ea36 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.549322 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112254eb-d87e-44d7-8448-646e2597ea36 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.549596 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112254eb-d87e-44d7-8448-646e2597ea36 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.551515 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 158/342] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:19.554781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112254eb-d87e-44d7-8448-646e2597ea36 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:19.576995 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f24ab1-f527-4d8d-8ab8-0151b24abbc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:19.578504 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f24ab1-f527-4d8d-8ab8-0151b24abbc3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:19.683652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 185/343] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:36:19.684982 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7331543-06a3-48f3-80e3-e609aba7591f None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.685331 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7331543-06a3-48f3-80e3-e609aba7591f None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.685667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7331543-06a3-48f3-80e3-e609aba7591f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.710236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-d7331543-06a3-48f3-80e3-e609aba7591f None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:19.711562 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-d7331543-06a3-48f3-80e3-e609aba7591f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:19.755169 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:19.771458 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112254eb-d87e-44d7-8448-646e2597ea36 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:19.798723 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 186/344] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:19.799943 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7331543-06a3-48f3-80e3-e609aba7591f None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:19.823769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a5812-b2ff-401b-98b8-b5392bcd3d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:19.824409 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a5812-b2ff-401b-98b8-b5392bcd3d14 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.824668 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a5812-b2ff-401b-98b8-b5392bcd3d14 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.825170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a5812-b2ff-401b-98b8-b5392bcd3d14 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.826294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a5812-b2ff-401b-98b8-b5392bcd3d14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:19.826794 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a5812-b2ff-401b-98b8-b5392bcd3d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:19.828394 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a5812-b2ff-401b-98b8-b5392bcd3d14 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:19.845674 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 159/345] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:19.846924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31637b28-00c3-4ca2-b624-1e86c3e04a7a None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.847227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31637b28-00c3-4ca2-b624-1e86c3e04a7a None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.847499 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31637b28-00c3-4ca2-b624-1e86c3e04a7a None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.848936 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 160/346] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:19.850855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31637b28-00c3-4ca2-b624-1e86c3e04a7a None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:19.907701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7331543-06a3-48f3-80e3-e609aba7591f None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:19.921597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:19.948349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe2add-d64f-4127-8708-e318f44caca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:19.949340 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe2add-d64f-4127-8708-e318f44caca7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.949829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe2add-d64f-4127-8708-e318f44caca7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.950288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe2add-d64f-4127-8708-e318f44caca7 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.951692 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe2add-d64f-4127-8708-e318f44caca7 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:19.953516 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe2add-d64f-4127-8708-e318f44caca7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:19.955987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 187/347] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:19 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:19.957356 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68326dce-6854-4073-ae3e-cbb16dd986ef None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:19.957836 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68326dce-6854-4073-ae3e-cbb16dd986ef None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:19.958293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68326dce-6854-4073-ae3e-cbb16dd986ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:19.981584 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-68326dce-6854-4073-ae3e-cbb16dd986ef None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:19.983100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-68326dce-6854-4073-ae3e-cbb16dd986ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:20.007822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31637b28-00c3-4ca2-b624-1e86c3e04a7a None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:20.015813 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:20.057415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 188/348] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:20.059071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68326dce-6854-4073-ae3e-cbb16dd986ef None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:20.062754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c6c65-6906-4e00-aa70-0514433061c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:20.063708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c6c65-6906-4e00-aa70-0514433061c9 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:20.064177 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c6c65-6906-4e00-aa70-0514433061c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:20.064630 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c6c65-6906-4e00-aa70-0514433061c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:20.170407 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68326dce-6854-4073-ae3e-cbb16dd986ef None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:20.206695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3629-57c3-4419-b995-254c21595edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:20.207738 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3629-57c3-4419-b995-254c21595edc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:20.208211 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3629-57c3-4419-b995-254c21595edc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:20.208705 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3629-57c3-4419-b995-254c21595edc None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:20.215506 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b3629-57c3-4419-b995-254c21595edc None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:20.217233 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b3629-57c3-4419-b995-254c21595edc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:20.220268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 189/349] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:20 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:20.228454 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3629-57c3-4419-b995-254c21595edc None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:20.251980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c6c65-6906-4e00-aa70-0514433061c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:20.253495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c6c65-6906-4e00-aa70-0514433061c9 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:20.265516 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:20.347990 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3629-57c3-4419-b995-254c21595edc None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:20.363477 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:20.388855 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7ce60-e599-4a44-9105-b4fa9e0d92e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:20.389817 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7ce60-e599-4a44-9105-b4fa9e0d92e8 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:20.390282 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7ce60-e599-4a44-9105-b4fa9e0d92e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:20.390736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7ce60-e599-4a44-9105-b4fa9e0d92e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:20.391844 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7ce60-e599-4a44-9105-b4fa9e0d92e8 None admin] RBAC: Adding query filter params (group.id=c9f8172ee7854d5ab716cfd558214b67, role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:20.392339 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7ce60-e599-4a44-9105-b4fa9e0d92e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:20.394162 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7ce60-e599-4a44-9105-b4fa9e0d92e8 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:20.400907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 190/350] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:20 2022] GET /identity/v3/role_assignments?group.id=c9f8172ee7854d5ab716cfd558214b67&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:20.475046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 161/351] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:36:20.476379 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7ce60-e599-4a44-9105-b4fa9e0d92e8 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:20.497888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c6c65-6906-4e00-aa70-0514433061c9 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:20.607245 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:20.616307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7ce60-e599-4a44-9105-b4fa9e0d92e8 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:20.652163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c6c65-6906-4e00-aa70-0514433061c9 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:20.669976 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc42ad-c440-4b5f-86ae-c052c1df53c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:20.670907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc42ad-c440-4b5f-86ae-c052c1df53c7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:20.671471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc42ad-c440-4b5f-86ae-c052c1df53c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:20.671907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc42ad-c440-4b5f-86ae-c052c1df53c7 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:20.673305 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc42ad-c440-4b5f-86ae-c052c1df53c7 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:20.674925 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc42ad-c440-4b5f-86ae-c052c1df53c7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:20.677101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 191/352] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:20 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:20.678057 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc42ad-c440-4b5f-86ae-c052c1df53c7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:20.701963 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd608d-16b9-44b9-84fa-2a0c6e525803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:20.702600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd608d-16b9-44b9-84fa-2a0c6e525803 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:20.702874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd608d-16b9-44b9-84fa-2a0c6e525803 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:20.703174 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd608d-16b9-44b9-84fa-2a0c6e525803 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:20.718371 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd608d-16b9-44b9-84fa-2a0c6e525803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-d4bd608d-16b9-44b9-84fa-2a0c6e525803 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:20.725813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:20.726755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:20.727613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 162/353] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:20.728530 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe495a-6796-4e7c-b75a-1f031b55a08d None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:20.728808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe495a-6796-4e7c-b75a-1f031b55a08d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:20.729078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe495a-6796-4e7c-b75a-1f031b55a08d None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:20.731817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 163/354] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:20.732542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe495a-6796-4e7c-b75a-1f031b55a08d None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:20.744826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:20.810583 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc42ad-c440-4b5f-86ae-c052c1df53c7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:20.855179 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9966e-824e-419e-8c6a-82ea2837eed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:20.856120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9966e-824e-419e-8c6a-82ea2837eed3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:20.856618 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9966e-824e-419e-8c6a-82ea2837eed3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:20.857054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9966e-824e-419e-8c6a-82ea2837eed3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:20.872497 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe495a-6796-4e7c-b75a-1f031b55a08d None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:20.873532 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9966e-824e-419e-8c6a-82ea2837eed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-21a9966e-824e-419e-8c6a-82ea2837eed3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:20.880661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:20.881775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:20.882754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:20.886215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 192/355] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:20.887213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9966e-824e-419e-8c6a-82ea2837eed3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:20.922306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:20.923839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e4a49-b2ca-43af-b4a9-4ad5d8b806c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:20.924740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e4a49-b2ca-43af-b4a9-4ad5d8b806c2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:20.925107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e4a49-b2ca-43af-b4a9-4ad5d8b806c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:20.925465 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e4a49-b2ca-43af-b4a9-4ad5d8b806c2 None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:20.928527 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e4a49-b2ca-43af-b4a9-4ad5d8b806c2 None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:20.930338 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e4a49-b2ca-43af-b4a9-4ad5d8b806c2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:20.933245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 164/356] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:20 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:20.934517 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14debc2a-a038-47f8-9000-58de8f352751 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:20.934977 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14debc2a-a038-47f8-9000-58de8f352751 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:20.940424 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14debc2a-a038-47f8-9000-58de8f352751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:20.965575 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-14debc2a-a038-47f8-9000-58de8f352751 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:20.967080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-14debc2a-a038-47f8-9000-58de8f352751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:21.031888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9966e-824e-419e-8c6a-82ea2837eed3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:21.066195 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 165/357] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:21.068216 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14debc2a-a038-47f8-9000-58de8f352751 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:21.076063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa01e93-c188-4b30-aeaa-b413b9272839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:21.076620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:21.077284 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa01e93-c188-4b30-aeaa-b413b9272839 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:21.077561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa01e93-c188-4b30-aeaa-b413b9272839 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:21.078039 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa01e93-c188-4b30-aeaa-b413b9272839 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:21.078712 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa01e93-c188-4b30-aeaa-b413b9272839 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:21.079498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa01e93-c188-4b30-aeaa-b413b9272839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:21.081329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa01e93-c188-4b30-aeaa-b413b9272839 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:21.092961 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 193/358] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:21.094124 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa01e93-c188-4b30-aeaa-b413b9272839 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:21.192473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:21.196064 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14debc2a-a038-47f8-9000-58de8f352751 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:21.218760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa01e93-c188-4b30-aeaa-b413b9272839 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:21.237076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244350b-a749-4ce9-a928-20954d8324d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:21.237076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244350b-a749-4ce9-a928-20954d8324d5 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:21.237367 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244350b-a749-4ce9-a928-20954d8324d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:21.237441 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244350b-a749-4ce9-a928-20954d8324d5 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:21.239006 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244350b-a749-4ce9-a928-20954d8324d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:21.239006 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244350b-a749-4ce9-a928-20954d8324d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:21.239854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244350b-a749-4ce9-a928-20954d8324d5 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:21.250690 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 166/359] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:21.251756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244350b-a749-4ce9-a928-20954d8324d5 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:21.339944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee6c4b-627b-4ee3-9fe0-ac3806768593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:21.340499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee6c4b-627b-4ee3-9fe0-ac3806768593 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:21.340780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee6c4b-627b-4ee3-9fe0-ac3806768593 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:21.341051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee6c4b-627b-4ee3-9fe0-ac3806768593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:21.341788 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee6c4b-627b-4ee3-9fe0-ac3806768593 None admin] RBAC: Adding query filter params (group.id=c9f8172ee7854d5ab716cfd558214b67, role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:21.342090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee6c4b-627b-4ee3-9fe0-ac3806768593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:21.343639 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee6c4b-627b-4ee3-9fe0-ac3806768593 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:21.365087 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 194/360] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:21 2022] GET /identity/v3/role_assignments?group.id=c9f8172ee7854d5ab716cfd558214b67&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 260 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:21.365877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee6c4b-627b-4ee3-9fe0-ac3806768593 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:21.367996 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:21.378986 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244350b-a749-4ce9-a928-20954d8324d5 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:21.456642 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ad4ea-0af8-4009-9375-5ca67679a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:21.457532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ad4ea-0af8-4009-9375-5ca67679a0ab None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:21.457897 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ad4ea-0af8-4009-9375-5ca67679a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:21.458271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ad4ea-0af8-4009-9375-5ca67679a0ab None admin] PATH_INFO: `/v3/users/f24df2764f884a9ea42f0deddea2de42` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:21.460057 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ad4ea-0af8-4009-9375-5ca67679a0ab None admin] RBAC: Authorizing `identity:get_user(user_id=f24df2764f884a9ea42f0deddea2de42)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:21.461725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ad4ea-0af8-4009-9375-5ca67679a0ab None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:21.463713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 167/361] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:36:21 2022] GET /identity/v3/users/f24df2764f884a9ea42f0deddea2de42 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:21.470173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ad4ea-0af8-4009-9375-5ca67679a0ab None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:21.523995 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee6c4b-627b-4ee3-9fe0-ac3806768593 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:21.574559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740850bd-3f2b-4c51-88a8-ddb399463a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:21.575216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740850bd-3f2b-4c51-88a8-ddb399463a0a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:21.575491 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740850bd-3f2b-4c51-88a8-ddb399463a0a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:21.575755 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740850bd-3f2b-4c51-88a8-ddb399463a0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:21.600083 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:21.621698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ad4ea-0af8-4009-9375-5ca67679a0ab None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:21.657669 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740850bd-3f2b-4c51-88a8-ddb399463a0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:21.659949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740850bd-3f2b-4c51-88a8-ddb399463a0a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:21.661790 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:21.669280 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee4cbd-9ab6-4836-b990-18bcd2b59369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:21.669890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee4cbd-9ab6-4836-b990-18bcd2b59369 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:21.670160 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee4cbd-9ab6-4836-b990-18bcd2b59369 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:21.670425 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee4cbd-9ab6-4836-b990-18bcd2b59369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:21.671172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee4cbd-9ab6-4836-b990-18bcd2b59369 None admin] RBAC: Adding query filter params (role.id=1ece0be4f4c9430b9909bc8d8e87adc3, user.id=f24df2764f884a9ea42f0deddea2de42, scope.system=all) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:21.671475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee4cbd-9ab6-4836-b990-18bcd2b59369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:21.673385 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee4cbd-9ab6-4836-b990-18bcd2b59369 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:21.690381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 168/362] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Aug 30 11:36:21 2022] GET /identity/v3/role_assignments?user.id=f24df2764f884a9ea42f0deddea2de42&scope.system=all&role.id=1ece0be4f4c9430b9909bc8d8e87adc3 => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:21.691255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee4cbd-9ab6-4836-b990-18bcd2b59369 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:21.786691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 195/363] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:21.788621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7b1d3-b01d-4578-9f2c-bf9ef637ba80 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:21.789134 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7b1d3-b01d-4578-9f2c-bf9ef637ba80 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:21.789610 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7b1d3-b01d-4578-9f2c-bf9ef637ba80 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:21.791810 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 196/364] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:21.792906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7b1d3-b01d-4578-9f2c-bf9ef637ba80 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:21.818962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee4cbd-9ab6-4836-b990-18bcd2b59369 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:21.861175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:21.861581 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fa720-6db4-48c6-9dfa-914caada68a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:21.862548 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fa720-6db4-48c6-9dfa-914caada68a7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:21.862823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fa720-6db4-48c6-9dfa-914caada68a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:21.863090 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fa720-6db4-48c6-9dfa-914caada68a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:21.877192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220fa720-6db4-48c6-9dfa-914caada68a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-220fa720-6db4-48c6-9dfa-914caada68a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:21.882760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:21.883746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:21.884556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:21.887014 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 169/365] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:36:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:21.888725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b54a9-99f6-467a-9e1e-fc22d8ceaf06 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:21.889198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b54a9-99f6-467a-9e1e-fc22d8ceaf06 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:21.889659 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b54a9-99f6-467a-9e1e-fc22d8ceaf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:21.912925 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-818b54a9-99f6-467a-9e1e-fc22d8ceaf06 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:21.914458 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-818b54a9-99f6-467a-9e1e-fc22d8ceaf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:21.939879 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7b1d3-b01d-4578-9f2c-bf9ef637ba80 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:21.984608 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:22.012579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 170/366] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:22.012999 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3236351-94ff-4655-acde-0c59696b3a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:22.013444 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b54a9-99f6-467a-9e1e-fc22d8ceaf06 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:22.013678 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3236351-94ff-4655-acde-0c59696b3a1d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:22.013979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3236351-94ff-4655-acde-0c59696b3a1d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:22.014248 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3236351-94ff-4655-acde-0c59696b3a1d None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:22.015400 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3236351-94ff-4655-acde-0c59696b3a1d None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:22.016920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3236351-94ff-4655-acde-0c59696b3a1d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:22.018843 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 197/367] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:21 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:22.026130 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3236351-94ff-4655-acde-0c59696b3a1d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:22.160727 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b54a9-99f6-467a-9e1e-fc22d8ceaf06 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:22.168562 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3236351-94ff-4655-acde-0c59696b3a1d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:22.197076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:22.225517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee62f7-52de-4279-83e5-fb4badee1562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:22.226172 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee62f7-52de-4279-83e5-fb4badee1562 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:22.226447 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee62f7-52de-4279-83e5-fb4badee1562 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:22.226712 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee62f7-52de-4279-83e5-fb4badee1562 None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:22.227859 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee62f7-52de-4279-83e5-fb4badee1562 None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:22.229387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee62f7-52de-4279-83e5-fb4badee1562 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:22.231330 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 198/368] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:22 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:22.237177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee62f7-52de-4279-83e5-fb4badee1562 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:22.239881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0a836-7ce5-4f61-bcae-1fa59479361b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:22.240518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a836-7ce5-4f61-bcae-1fa59479361b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:22.240793 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a836-7ce5-4f61-bcae-1fa59479361b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:22.241061 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a836-7ce5-4f61-bcae-1fa59479361b None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:22.241743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0a836-7ce5-4f61-bcae-1fa59479361b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:22.242041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0a836-7ce5-4f61-bcae-1fa59479361b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:22.243459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0a836-7ce5-4f61-bcae-1fa59479361b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:22.265364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 171/369] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:22.270479 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0a836-7ce5-4f61-bcae-1fa59479361b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:22.350949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee62f7-52de-4279-83e5-fb4badee1562 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:22.377927 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:22.395577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3075-95b1-4b47-9346-c7c174ba8c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:22.396306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3075-95b1-4b47-9346-c7c174ba8c32 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:22.396644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3075-95b1-4b47-9346-c7c174ba8c32 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:22.396976 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3075-95b1-4b47-9346-c7c174ba8c32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:22.398030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3075-95b1-4b47-9346-c7c174ba8c32 None admin] RBAC: Adding query filter params (group.id=c9f8172ee7854d5ab716cfd558214b67, role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=d7a7423471544bcb8851a5df381c6144) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:22.398745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3075-95b1-4b47-9346-c7c174ba8c32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:22.400555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3075-95b1-4b47-9346-c7c174ba8c32 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:22.415236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 199/370] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:22 2022] GET /identity/v3/role_assignments?group.id=c9f8172ee7854d5ab716cfd558214b67&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:22.415236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3075-95b1-4b47-9346-c7c174ba8c32 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:22.479953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:22.489737 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0a836-7ce5-4f61-bcae-1fa59479361b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:22.537899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3075-95b1-4b47-9346-c7c174ba8c32 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:22.579816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fdbb8-2f92-4537-94b5-8420325c2d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:22.580604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fdbb8-2f92-4537-94b5-8420325c2d6e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:22.580902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fdbb8-2f92-4537-94b5-8420325c2d6e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:22.581241 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fdbb8-2f92-4537-94b5-8420325c2d6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:22.586319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de622b-d368-4d5b-8dd3-22947c6ba22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:22.587219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de622b-d368-4d5b-8dd3-22947c6ba22b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:22.587670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de622b-d368-4d5b-8dd3-22947c6ba22b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:22.588308 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de622b-d368-4d5b-8dd3-22947c6ba22b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:22.594382 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fdbb8-2f92-4537-94b5-8420325c2d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-e01fdbb8-2f92-4537-94b5-8420325c2d6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:22.600364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:22.601281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:22.602103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:22.602932 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 200/371] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:36:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:22.608143 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fdbb8-2f92-4537-94b5-8420325c2d6e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:22.704356 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de622b-d368-4d5b-8dd3-22947c6ba22b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:22.706511 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de622b-d368-4d5b-8dd3-22947c6ba22b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:22.735200 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fdbb8-2f92-4537-94b5-8420325c2d6e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:22.747540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:22.772241 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de1ee5-f628-4121-aaa3-f49a9808cb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:22.772837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de1ee5-f628-4121-aaa3-f49a9808cb7e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:22.773110 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de1ee5-f628-4121-aaa3-f49a9808cb7e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:22.773375 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de1ee5-f628-4121-aaa3-f49a9808cb7e None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:22.774975 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de1ee5-f628-4121-aaa3-f49a9808cb7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:22.775316 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de1ee5-f628-4121-aaa3-f49a9808cb7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:22.776708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de1ee5-f628-4121-aaa3-f49a9808cb7e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:22.786617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 201/372] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:22.791841 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de1ee5-f628-4121-aaa3-f49a9808cb7e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:22.794004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:22.902403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de1ee5-f628-4121-aaa3-f49a9808cb7e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:22.942004 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a39cd-3872-4bfd-8959-5ea51f0818f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:22.942614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a39cd-3872-4bfd-8959-5ea51f0818f2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:22.942953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a39cd-3872-4bfd-8959-5ea51f0818f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:22.943243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a39cd-3872-4bfd-8959-5ea51f0818f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:22.950296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 172/373] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:36:22.952452 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541565-c601-48cb-9133-f1ff895b5220 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:22.952946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541565-c601-48cb-9133-f1ff895b5220 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:22.953410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541565-c601-48cb-9133-f1ff895b5220 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:22.960628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 173/374] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:22.962072 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74541565-c601-48cb-9133-f1ff895b5220 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:22.996531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:23.034519 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a39cd-3872-4bfd-8959-5ea51f0818f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:23.037194 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a39cd-3872-4bfd-8959-5ea51f0818f2 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:23.088711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74541565-c601-48cb-9133-f1ff895b5220 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:23.124996 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a2d02-5e13-4878-a602-5193a7c231a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:23.125635 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a2d02-5e13-4878-a602-5193a7c231a3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.125902 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a2d02-5e13-4878-a602-5193a7c231a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.126164 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a2d02-5e13-4878-a602-5193a7c231a3 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.131983 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a2d02-5e13-4878-a602-5193a7c231a3 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:23.131983 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a2d02-5e13-4878-a602-5193a7c231a3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:23.136356 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 174/375] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:22 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:23.137579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6773ef1-fa91-46c3-b4d7-73a8445b615e None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.137854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6773ef1-fa91-46c3-b4d7-73a8445b615e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.138118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6773ef1-fa91-46c3-b4d7-73a8445b615e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.169576 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-c6773ef1-fa91-46c3-b4d7-73a8445b615e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:23.171460 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c6773ef1-fa91-46c3-b4d7-73a8445b615e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:23.176174 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:23.182125 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 202/376] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:23.182771 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a39cd-3872-4bfd-8959-5ea51f0818f2 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:23.260923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 175/377] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:23.262080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6773ef1-fa91-46c3-b4d7-73a8445b615e None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:23.274492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:23.304594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a39cd-3872-4bfd-8959-5ea51f0818f2 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:23.344210 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e306ed5-0d97-4029-8a44-fcd31c4a1e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:23.344938 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e306ed5-0d97-4029-8a44-fcd31c4a1e18 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.345270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e306ed5-0d97-4029-8a44-fcd31c4a1e18 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.345596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e306ed5-0d97-4029-8a44-fcd31c4a1e18 None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.347486 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e306ed5-0d97-4029-8a44-fcd31c4a1e18 None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:23.348953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e306ed5-0d97-4029-8a44-fcd31c4a1e18 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:23.350535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 203/378] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:23 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:23.357957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e306ed5-0d97-4029-8a44-fcd31c4a1e18 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:23.394023 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6773ef1-fa91-46c3-b4d7-73a8445b615e None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:23.442011 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142a5d1-c922-45c1-a002-0a9d4e511cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:23.442870 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142a5d1-c922-45c1-a002-0a9d4e511cc9 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.443317 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142a5d1-c922-45c1-a002-0a9d4e511cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.443721 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142a5d1-c922-45c1-a002-0a9d4e511cc9 None admin] PATH_INFO: `/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.445536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142a5d1-c922-45c1-a002-0a9d4e511cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=a2579099690b4b9ea0cc6cb7695b8b5a)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:23.447213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142a5d1-c922-45c1-a002-0a9d4e511cc9 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:23.449732 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 176/379] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:23 2022] GET /identity/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:23.454955 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142a5d1-c922-45c1-a002-0a9d4e511cc9 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:23.467505 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:23.539148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e306ed5-0d97-4029-8a44-fcd31c4a1e18 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:23.585942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2b39-fe42-4878-8a01-82e2f99ebd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:23.586558 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2b39-fe42-4878-8a01-82e2f99ebd50 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.586830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2b39-fe42-4878-8a01-82e2f99ebd50 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.587091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2b39-fe42-4878-8a01-82e2f99ebd50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.587899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd2b39-fe42-4878-8a01-82e2f99ebd50 None admin] RBAC: Adding query filter params (group.id=c9f8172ee7854d5ab716cfd558214b67, role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=d7a7423471544bcb8851a5df381c6144) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:23.588171 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd2b39-fe42-4878-8a01-82e2f99ebd50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:23.589609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd2b39-fe42-4878-8a01-82e2f99ebd50 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:23.596476 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 204/380] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:23 2022] GET /identity/v3/role_assignments?group.id=c9f8172ee7854d5ab716cfd558214b67&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:23.599543 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142a5d1-c922-45c1-a002-0a9d4e511cc9 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:23.659894 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccacb4-3c19-4785-849f-7c11308a4f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:23.660552 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2b39-fe42-4878-8a01-82e2f99ebd50 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:23.661216 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccacb4-3c19-4785-849f-7c11308a4f06 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.661787 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccacb4-3c19-4785-849f-7c11308a4f06 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.662235 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccacb4-3c19-4785-849f-7c11308a4f06 None admin] PATH_INFO: `/v3/groups/0a231dffa2dd4db7bcad94baf472b066` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.665149 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:23.678460 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccacb4-3c19-4785-849f-7c11308a4f06 None admin] RBAC: Authorizing `identity:get_group(group_id=0a231dffa2dd4db7bcad94baf472b066)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:23.680129 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccacb4-3c19-4785-849f-7c11308a4f06 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:23.683302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 177/381] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:23 2022] GET /identity/v3/groups/0a231dffa2dd4db7bcad94baf472b066 => generated 229 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:23.688988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccacb4-3c19-4785-849f-7c11308a4f06 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:23.802035 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:23.821803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2b39-fe42-4878-8a01-82e2f99ebd50 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:23.865302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccacb4-3c19-4785-849f-7c11308a4f06 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:23.879827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013fbb9-426b-484f-936b-1ea24b0dcb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:23.881046 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013fbb9-426b-484f-936b-1ea24b0dcb8a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.881464 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013fbb9-426b-484f-936b-1ea24b0dcb8a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.881902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013fbb9-426b-484f-936b-1ea24b0dcb8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.896571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013fbb9-426b-484f-936b-1ea24b0dcb8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-8013fbb9-426b-484f-936b-1ea24b0dcb8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:23.904621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:23.905765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:23.906798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:23.909345 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 205/382] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:23.910138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e67efb-5afb-4a9a-aee0-fd86b494566e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:23.910883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e67efb-5afb-4a9a-aee0-fd86b494566e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.911138 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029c1c2-641d-43f0-981c-2c488e2ffe66 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.911282 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e67efb-5afb-4a9a-aee0-fd86b494566e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.911755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e67efb-5afb-4a9a-aee0-fd86b494566e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.911951 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029c1c2-641d-43f0-981c-2c488e2ffe66 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.912375 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029c1c2-641d-43f0-981c-2c488e2ffe66 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.912690 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e67efb-5afb-4a9a-aee0-fd86b494566e None admin] RBAC: Adding query filter params (group.id=0a231dffa2dd4db7bcad94baf472b066, role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=a2579099690b4b9ea0cc6cb7695b8b5a) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:23.913053 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e67efb-5afb-4a9a-aee0-fd86b494566e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:23.914396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 206/383] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:23.914710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e67efb-5afb-4a9a-aee0-fd86b494566e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:23.915483 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029c1c2-641d-43f0-981c-2c488e2ffe66 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:23.924978 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 178/384] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:23 2022] GET /identity/v3/role_assignments?group.id=0a231dffa2dd4db7bcad94baf472b066&scope.project.id=a2579099690b4b9ea0cc6cb7695b8b5a&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:23.930277 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a9814-edfa-4746-be8e-9ee8ab17223d None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:23.931701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a9814-edfa-4746-be8e-9ee8ab17223d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:23.932206 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a9814-edfa-4746-be8e-9ee8ab17223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:23.961747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-c51a9814-edfa-4746-be8e-9ee8ab17223d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:23.963517 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c51a9814-edfa-4746-be8e-9ee8ab17223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:24.054172 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:24.093594 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 179/385] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:24.111387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029c1c2-641d-43f0-981c-2c488e2ffe66 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:24.149576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64079cb-1829-4b81-863a-7365da835e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:24.150565 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64079cb-1829-4b81-863a-7365da835e16 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:24.151056 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64079cb-1829-4b81-863a-7365da835e16 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:24.151638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64079cb-1829-4b81-863a-7365da835e16 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:24.152548 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64079cb-1829-4b81-863a-7365da835e16 None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:24.153051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64079cb-1829-4b81-863a-7365da835e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:24.154746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64079cb-1829-4b81-863a-7365da835e16 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:24.165032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 207/386] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:24.170455 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a9814-edfa-4746-be8e-9ee8ab17223d None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:24.287404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:24.323225 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a9814-edfa-4746-be8e-9ee8ab17223d None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:24.360817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabcaf7-e70b-441d-815f-1166b79fccc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:24.361444 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabcaf7-e70b-441d-815f-1166b79fccc2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:24.361715 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabcaf7-e70b-441d-815f-1166b79fccc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:24.361977 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabcaf7-e70b-441d-815f-1166b79fccc2 None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:24.363136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabcaf7-e70b-441d-815f-1166b79fccc2 None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:24.364621 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabcaf7-e70b-441d-815f-1166b79fccc2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:24.366193 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 180/387] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:24 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:24.372074 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64079cb-1829-4b81-863a-7365da835e16 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:24.426474 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabcaf7-e70b-441d-815f-1166b79fccc2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:24.507361 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64079cb-1829-4b81-863a-7365da835e16 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:24.539463 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:24.547839 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfebb44-bc00-4093-b859-ef2beec58acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:24.548487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfebb44-bc00-4093-b859-ef2beec58acc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:24.548759 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfebb44-bc00-4093-b859-ef2beec58acc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:24.549023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfebb44-bc00-4093-b859-ef2beec58acc None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:24.550224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfebb44-bc00-4093-b859-ef2beec58acc None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:24.550374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabcaf7-e70b-441d-815f-1166b79fccc2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:24.552279 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfebb44-bc00-4093-b859-ef2beec58acc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:24.555837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 208/388] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:24 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:24.564807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfebb44-bc00-4093-b859-ef2beec58acc None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:24.603038 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e7e4d-6e0e-48ea-a696-55b408507d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:24.604204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e7e4d-6e0e-48ea-a696-55b408507d9e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:24.604693 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e7e4d-6e0e-48ea-a696-55b408507d9e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:24.605216 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e7e4d-6e0e-48ea-a696-55b408507d9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:24.608963 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:24.620146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e7e4d-6e0e-48ea-a696-55b408507d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-f72e7e4d-6e0e-48ea-a696-55b408507d9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:24.626044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:24.626917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:24.627848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:24.630736 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 181/389] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:36:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:24.636998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e7e4d-6e0e-48ea-a696-55b408507d9e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:24.688884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfebb44-bc00-4093-b859-ef2beec58acc None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:24.732705 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c0268-508d-4846-8c70-079b625e0e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:24.733850 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c0268-508d-4846-8c70-079b625e0e18 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:24.734138 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c0268-508d-4846-8c70-079b625e0e18 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:24.734405 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c0268-508d-4846-8c70-079b625e0e18 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/groups/c9f8172ee7854d5ab716cfd558214b67/roles/91e10f3d9d934fe1be9d3642b8092d08` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:24.736824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c0268-508d-4846-8c70-079b625e0e18 None admin] RBAC: Authorizing `identity:create_grant(project_id=de3741f1b798440b9e1a4f4b24dbd224, group_id=c9f8172ee7854d5ab716cfd558214b67, role_id=91e10f3d9d934fe1be9d3642b8092d08)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:24.738649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c0268-508d-4846-8c70-079b625e0e18 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:24.755886 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e7e4d-6e0e-48ea-a696-55b408507d9e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:24.759284 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 209/390] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Aug 30 11:36:24 2022] PUT /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/groups/c9f8172ee7854d5ab716cfd558214b67/roles/91e10f3d9d934fe1be9d3642b8092d08 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:24.810043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3002b9-4a3f-4087-8929-382e34c4a025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:24.810755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3002b9-4a3f-4087-8929-382e34c4a025 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:24.810935 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3002b9-4a3f-4087-8929-382e34c4a025 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:24.811221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3002b9-4a3f-4087-8929-382e34c4a025 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:24.811905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3002b9-4a3f-4087-8929-382e34c4a025 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:24.812202 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3002b9-4a3f-4087-8929-382e34c4a025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:24.813647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3002b9-4a3f-4087-8929-382e34c4a025 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:24.824284 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 182/391] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:24.830490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c0268-508d-4846-8c70-079b625e0e18 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:24.900651 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:24.960251 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c0268-508d-4846-8c70-079b625e0e18 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:25.006113 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb343c1-e3f0-4b6f-b8e7-604b7fe26ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:25.006762 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb343c1-e3f0-4b6f-b8e7-604b7fe26ad2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:25.007078 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb343c1-e3f0-4b6f-b8e7-604b7fe26ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:25.007445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb343c1-e3f0-4b6f-b8e7-604b7fe26ad2 None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:25.009536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb343c1-e3f0-4b6f-b8e7-604b7fe26ad2 None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:25.010923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb343c1-e3f0-4b6f-b8e7-604b7fe26ad2 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:25.012863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 210/392] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:24 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:25.018445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3002b9-4a3f-4087-8929-382e34c4a025 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:25.086844 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:25.133929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3002b9-4a3f-4087-8929-382e34c4a025 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:25.171813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dee3a-936a-4cc7-90d8-ba795353e170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:25.172376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dee3a-936a-4cc7-90d8-ba795353e170 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:25.172647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dee3a-936a-4cc7-90d8-ba795353e170 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:25.172916 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dee3a-936a-4cc7-90d8-ba795353e170 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:25.174061 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dee3a-936a-4cc7-90d8-ba795353e170 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:25.175569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dee3a-936a-4cc7-90d8-ba795353e170 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:25.177322 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 183/393] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:25 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:25.183148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb343c1-e3f0-4b6f-b8e7-604b7fe26ad2 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:25.285095 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb343c1-e3f0-4b6f-b8e7-604b7fe26ad2 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:25.313037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:25.320890 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493e0e8-7ac6-49da-8a61-97338b9adb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:25.321659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493e0e8-7ac6-49da-8a61-97338b9adb95 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:25.322187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493e0e8-7ac6-49da-8a61-97338b9adb95 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:25.322957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493e0e8-7ac6-49da-8a61-97338b9adb95 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/groups/c9f8172ee7854d5ab716cfd558214b67/roles/05aee5278bd743c4bb8fb605feb806a9` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:25.325457 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493e0e8-7ac6-49da-8a61-97338b9adb95 None admin] RBAC: Authorizing `identity:create_grant(project_id=de3741f1b798440b9e1a4f4b24dbd224, group_id=c9f8172ee7854d5ab716cfd558214b67, role_id=05aee5278bd743c4bb8fb605feb806a9)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:25.327342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493e0e8-7ac6-49da-8a61-97338b9adb95 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:25.342590 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 211/394] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Aug 30 11:36:25 2022] PUT /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224/groups/c9f8172ee7854d5ab716cfd558214b67/roles/05aee5278bd743c4bb8fb605feb806a9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:25.499902 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef954db1-1b23-4c36-a2f9-6ea4ee7edb69 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:25.500236 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef954db1-1b23-4c36-a2f9-6ea4ee7edb69 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:25.500434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef954db1-1b23-4c36-a2f9-6ea4ee7edb69 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:25.502050 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 184/395] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:25.507100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa010914-4dbe-40a8-b721-effa8cca5fa7 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:25.507466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa010914-4dbe-40a8-b721-effa8cca5fa7 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:25.507733 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa010914-4dbe-40a8-b721-effa8cca5fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:25.529174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-fa010914-4dbe-40a8-b721-effa8cca5fa7 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:25.530373 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-fa010914-4dbe-40a8-b721-effa8cca5fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:25.605792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 212/396] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:26.100168 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef954db1-1b23-4c36-a2f9-6ea4ee7edb69 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:26.206679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef954db1-1b23-4c36-a2f9-6ea4ee7edb69 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:26.239695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:26.241753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb1c58-fa8c-4b18-b03e-5b3aa5a67982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:26.242394 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb1c58-fa8c-4b18-b03e-5b3aa5a67982 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:26.242964 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb1c58-fa8c-4b18-b03e-5b3aa5a67982 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:26.243258 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb1c58-fa8c-4b18-b03e-5b3aa5a67982 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:26.254946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb1c58-fa8c-4b18-b03e-5b3aa5a67982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-7afb1c58-fa8c-4b18-b03e-5b3aa5a67982 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:26.260419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:26.261467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:26.262314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:26.265064 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 185/397] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:26.270983 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa010914-4dbe-40a8-b721-effa8cca5fa7 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:26.284265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ca60a-ab1a-46f5-920d-09a6aa661ca5 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:26.284570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ca60a-ab1a-46f5-920d-09a6aa661ca5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:26.284793 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ca60a-ab1a-46f5-920d-09a6aa661ca5 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:26.288308 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 186/398] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:26.304795 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e2e58-9f62-4b9d-b050-a6b53de34d11 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:26.305115 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e2e58-9f62-4b9d-b050-a6b53de34d11 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:26.305324 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e2e58-9f62-4b9d-b050-a6b53de34d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:26.341724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-cd3e2e58-9f62-4b9d-b050-a6b53de34d11 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:26.342941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-cd3e2e58-9f62-4b9d-b050-a6b53de34d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:26.423644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa010914-4dbe-40a8-b721-effa8cca5fa7 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:26.447480 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:26.463062 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fb3c4-e4f5-4551-b134-d0b9acc78d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:26.464001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fb3c4-e4f5-4551-b134-d0b9acc78d77 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:26.464320 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fb3c4-e4f5-4551-b134-d0b9acc78d77 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:26.464818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fb3c4-e4f5-4551-b134-d0b9acc78d77 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:26.465852 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fb3c4-e4f5-4551-b134-d0b9acc78d77 None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:26.466165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fb3c4-e4f5-4551-b134-d0b9acc78d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:26.467777 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fb3c4-e4f5-4551-b134-d0b9acc78d77 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:26.479268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 213/399] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:26.481140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e069-cbcd-4096-b656-e26c47ae6fec None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:26.481570 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e069-cbcd-4096-b656-e26c47ae6fec None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:26.482025 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e069-cbcd-4096-b656-e26c47ae6fec None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:26.483436 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 214/400] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:26.489368 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c75ff2-d37e-41dc-9422-4be935229197 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:26.489838 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c75ff2-d37e-41dc-9422-4be935229197 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:26.490203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c75ff2-d37e-41dc-9422-4be935229197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:26.492470 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 187/401] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:26.493309 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e2e58-9f62-4b9d-b050-a6b53de34d11 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:26.514897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-e1c75ff2-d37e-41dc-9422-4be935229197 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:26.516248 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-e1c75ff2-d37e-41dc-9422-4be935229197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:26.605205 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 215/402] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:26.617954 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e2e58-9f62-4b9d-b050-a6b53de34d11 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:26.633464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:26.654838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc696b41-7c69-42e0-8819-300f59a84c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:26.655503 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc696b41-7c69-42e0-8819-300f59a84c50 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:26.655772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc696b41-7c69-42e0-8819-300f59a84c50 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:26.656037 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc696b41-7c69-42e0-8819-300f59a84c50 None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:26.657161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc696b41-7c69-42e0-8819-300f59a84c50 None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:26.658598 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc696b41-7c69-42e0-8819-300f59a84c50 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:26.660495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 188/403] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:26 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:26.666040 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c75ff2-d37e-41dc-9422-4be935229197 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:26.767287 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c75ff2-d37e-41dc-9422-4be935229197 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:26.771468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:26.802936 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517666e-a7b0-441e-b157-ac1e1c45b360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:26.803645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517666e-a7b0-441e-b157-ac1e1c45b360 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:26.803920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517666e-a7b0-441e-b157-ac1e1c45b360 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:26.804181 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517666e-a7b0-441e-b157-ac1e1c45b360 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:26.805309 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517666e-a7b0-441e-b157-ac1e1c45b360 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:26.806752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517666e-a7b0-441e-b157-ac1e1c45b360 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:26.808696 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 216/404] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:26 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:26.814024 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc696b41-7c69-42e0-8819-300f59a84c50 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:26.922203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517666e-a7b0-441e-b157-ac1e1c45b360 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:26.926338 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc696b41-7c69-42e0-8819-300f59a84c50 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:26.963033 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef650fc-cfaa-4357-9071-c056240e8116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:26.963805 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef650fc-cfaa-4357-9071-c056240e8116 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:26.964080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef650fc-cfaa-4357-9071-c056240e8116 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:26.964350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef650fc-cfaa-4357-9071-c056240e8116 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144/groups/c9f8172ee7854d5ab716cfd558214b67/roles/91e10f3d9d934fe1be9d3642b8092d08` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:26.966337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef650fc-cfaa-4357-9071-c056240e8116 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7a7423471544bcb8851a5df381c6144, group_id=c9f8172ee7854d5ab716cfd558214b67, role_id=91e10f3d9d934fe1be9d3642b8092d08)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:26.967811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef650fc-cfaa-4357-9071-c056240e8116 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:26.985309 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 189/405] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Aug 30 11:36:26 2022] PUT /identity/v3/projects/d7a7423471544bcb8851a5df381c6144/groups/c9f8172ee7854d5ab716cfd558214b67/roles/91e10f3d9d934fe1be9d3642b8092d08 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 11:36:26.986815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef650fc-cfaa-4357-9071-c056240e8116 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:27.057443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517666e-a7b0-441e-b157-ac1e1c45b360 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:27.079188 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:27.107892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf4c5f-501b-4f19-834c-0cf1a994856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:27.109068 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf4c5f-501b-4f19-834c-0cf1a994856c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.109338 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf4c5f-501b-4f19-834c-0cf1a994856c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.109601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf4c5f-501b-4f19-834c-0cf1a994856c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.123625 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf4c5f-501b-4f19-834c-0cf1a994856c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-5dcf4c5f-501b-4f19-834c-0cf1a994856c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:27.129156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:27.130153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:27.131853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 217/406] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:36:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:27.137414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf4c5f-501b-4f19-834c-0cf1a994856c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:27.181574 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef650fc-cfaa-4357-9071-c056240e8116 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:27.220587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24185ef3-c9dd-4c1a-a4b9-183f85b6d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:27.221232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24185ef3-c9dd-4c1a-a4b9-183f85b6d967 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.221507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24185ef3-c9dd-4c1a-a4b9-183f85b6d967 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.221773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24185ef3-c9dd-4c1a-a4b9-183f85b6d967 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.223336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:27.234869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24185ef3-c9dd-4c1a-a4b9-183f85b6d967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-24185ef3-c9dd-4c1a-a4b9-183f85b6d967 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:27.239956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:27.240866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:27.242713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 190/407] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:36:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:27.249224 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24185ef3-c9dd-4c1a-a4b9-183f85b6d967 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:27.343852 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf4c5f-501b-4f19-834c-0cf1a994856c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:27.380616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24185ef3-c9dd-4c1a-a4b9-183f85b6d967 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:27.382146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:27.392441 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538af47-7ccc-4b7d-9f38-5a87951a8129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:27.393064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538af47-7ccc-4b7d-9f38-5a87951a8129 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.393339 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538af47-7ccc-4b7d-9f38-5a87951a8129 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.393608 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538af47-7ccc-4b7d-9f38-5a87951a8129 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.394291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538af47-7ccc-4b7d-9f38-5a87951a8129 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:27.394594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538af47-7ccc-4b7d-9f38-5a87951a8129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:27.396324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538af47-7ccc-4b7d-9f38-5a87951a8129 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:27.408688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 218/408] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:27.410066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a6e66-ecde-4f3f-80a8-4dab6ba34b25 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.410444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a6e66-ecde-4f3f-80a8-4dab6ba34b25 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.410718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a6e66-ecde-4f3f-80a8-4dab6ba34b25 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.416358 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 219/409] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:27.417765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a6e66-ecde-4f3f-80a8-4dab6ba34b25 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:27.442527 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e5df3-bf6a-4618-9c7d-f1593e001a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:27.443387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e5df3-bf6a-4618-9c7d-f1593e001a86 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.443983 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e5df3-bf6a-4618-9c7d-f1593e001a86 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.444325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e5df3-bf6a-4618-9c7d-f1593e001a86 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.445460 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e5df3-bf6a-4618-9c7d-f1593e001a86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:27.445839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e5df3-bf6a-4618-9c7d-f1593e001a86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:27.447393 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e5df3-bf6a-4618-9c7d-f1593e001a86 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:27.459001 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 191/410] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:27.460636 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f26fe-dc52-447a-a899-f9034b59c661 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.460994 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f26fe-dc52-447a-a899-f9034b59c661 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.461333 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f26fe-dc52-447a-a899-f9034b59c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.480271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-e31f26fe-dc52-447a-a899-f9034b59c661 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:27.481697 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-e31f26fe-dc52-447a-a899-f9034b59c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:27.534715 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:27.543366 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a6e66-ecde-4f3f-80a8-4dab6ba34b25 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:27.556478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 192/411] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:27.564052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f26fe-dc52-447a-a899-f9034b59c661 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:27.580510 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962e347-57bc-4b4b-b302-ef3b4e587329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:27.582598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962e347-57bc-4b4b-b302-ef3b4e587329 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.583324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962e347-57bc-4b4b-b302-ef3b4e587329 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.584426 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962e347-57bc-4b4b-b302-ef3b4e587329 None admin] PATH_INFO: `/v3/groups/0a231dffa2dd4db7bcad94baf472b066` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.587225 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962e347-57bc-4b4b-b302-ef3b4e587329 None admin] RBAC: Authorizing `identity:get_group(group_id=0a231dffa2dd4db7bcad94baf472b066)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:27.589035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962e347-57bc-4b4b-b302-ef3b4e587329 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:27.592536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 220/412] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:27 2022] GET /identity/v3/groups/0a231dffa2dd4db7bcad94baf472b066 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:27.598420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962e347-57bc-4b4b-b302-ef3b4e587329 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:27.732458 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:27.743316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f26fe-dc52-447a-a899-f9034b59c661 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:27.761512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962e347-57bc-4b4b-b302-ef3b4e587329 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:27.795681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d788e-0990-4183-b9af-54605fec267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:27.796614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d788e-0990-4183-b9af-54605fec267c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.797033 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d788e-0990-4183-b9af-54605fec267c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.797500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d788e-0990-4183-b9af-54605fec267c None admin] PATH_INFO: `/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.798785 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d788e-0990-4183-b9af-54605fec267c None admin] RBAC: Authorizing `identity:get_project(project_id=a2579099690b4b9ea0cc6cb7695b8b5a)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:27.800558 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d788e-0990-4183-b9af-54605fec267c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:27.802856 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 221/413] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:27 2022] GET /identity/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:27.808835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d788e-0990-4183-b9af-54605fec267c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:27.839839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c5fc3-0489-45fb-b7b2-07fee09ec0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:27.840493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c5fc3-0489-45fb-b7b2-07fee09ec0cd None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.840760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c5fc3-0489-45fb-b7b2-07fee09ec0cd None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.841025 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c5fc3-0489-45fb-b7b2-07fee09ec0cd None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.841383 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:27.842224 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c5fc3-0489-45fb-b7b2-07fee09ec0cd None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:27.843693 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c5fc3-0489-45fb-b7b2-07fee09ec0cd None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:27.845618 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 193/414] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:27 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:27.852086 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c5fc3-0489-45fb-b7b2-07fee09ec0cd None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:27.921789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d788e-0990-4183-b9af-54605fec267c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:27.958687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c5fc3-0489-45fb-b7b2-07fee09ec0cd None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:27.958877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c1e7d-fd96-4097-996f-2d7f1da0543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:27.959595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c1e7d-fd96-4097-996f-2d7f1da0543b None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.959874 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c1e7d-fd96-4097-996f-2d7f1da0543b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.960237 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c1e7d-fd96-4097-996f-2d7f1da0543b None admin] PATH_INFO: `/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a/groups/0a231dffa2dd4db7bcad94baf472b066/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.962242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c1e7d-fd96-4097-996f-2d7f1da0543b None admin] RBAC: Authorizing `identity:create_grant(project_id=a2579099690b4b9ea0cc6cb7695b8b5a, group_id=0a231dffa2dd4db7bcad94baf472b066, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:27.963738 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c1e7d-fd96-4097-996f-2d7f1da0543b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:27.977434 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 222/415] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Aug 30 11:36:27 2022] PUT /identity/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a/groups/0a231dffa2dd4db7bcad94baf472b066/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:27.978602 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b5c3-8c03-4aa2-8206-69e463201112 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:27.978878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b5c3-8c03-4aa2-8206-69e463201112 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:27.979166 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b5c3-8c03-4aa2-8206-69e463201112 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:27.980581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 223/416] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:27.985440 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b5c3-8c03-4aa2-8206-69e463201112 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:28.025279 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd068c1-6bed-4a86-9a56-a87247ed09b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:28.026126 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd068c1-6bed-4a86-9a56-a87247ed09b7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:28.026417 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd068c1-6bed-4a86-9a56-a87247ed09b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:28.026836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd068c1-6bed-4a86-9a56-a87247ed09b7 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:28.028209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd068c1-6bed-4a86-9a56-a87247ed09b7 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:28.031213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd068c1-6bed-4a86-9a56-a87247ed09b7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:28.033668 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 194/417] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:27 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:28.034998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d5e30-00a6-4df6-a81d-ac210d29b8bc None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:28.035308 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d5e30-00a6-4df6-a81d-ac210d29b8bc None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:28.035579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d5e30-00a6-4df6-a81d-ac210d29b8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:28.061082 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-a86d5e30-00a6-4df6-a81d-ac210d29b8bc None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:28.062493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-a86d5e30-00a6-4df6-a81d-ac210d29b8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:28.090993 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:28.112229 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:28.132147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b5c3-8c03-4aa2-8206-69e463201112 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:28.151360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 195/418] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:28.152199 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d5e30-00a6-4df6-a81d-ac210d29b8bc None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:28.175276 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b79c49-329d-4bd2-aa13-b0f137b2421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:28.175929 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b79c49-329d-4bd2-aa13-b0f137b2421e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:28.176208 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b79c49-329d-4bd2-aa13-b0f137b2421e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:28.176476 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b79c49-329d-4bd2-aa13-b0f137b2421e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:28.190087 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b79c49-329d-4bd2-aa13-b0f137b2421e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-56b79c49-329d-4bd2-aa13-b0f137b2421e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:28.195530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:28.196522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:28.197638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 224/419] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:28.205460 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b79c49-329d-4bd2-aa13-b0f137b2421e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:28.355945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d5e30-00a6-4df6-a81d-ac210d29b8bc None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:28.398498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b79c49-329d-4bd2-aa13-b0f137b2421e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:28.427033 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067a5b7-e3a9-4b97-9d91-c2092d6e356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:28.428537 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a5b7-e3a9-4b97-9d91-c2092d6e356e None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:28.429205 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a5b7-e3a9-4b97-9d91-c2092d6e356e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:28.429910 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a5b7-e3a9-4b97-9d91-c2092d6e356e None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144/groups/c9f8172ee7854d5ab716cfd558214b67/roles/05aee5278bd743c4bb8fb605feb806a9` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:28.438968 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067a5b7-e3a9-4b97-9d91-c2092d6e356e None admin] RBAC: Authorizing `identity:create_grant(project_id=d7a7423471544bcb8851a5df381c6144, group_id=c9f8172ee7854d5ab716cfd558214b67, role_id=05aee5278bd743c4bb8fb605feb806a9)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:28.440759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067a5b7-e3a9-4b97-9d91-c2092d6e356e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:28.460223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 196/420] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Aug 30 11:36:28 2022] PUT /identity/v3/projects/d7a7423471544bcb8851a5df381c6144/groups/c9f8172ee7854d5ab716cfd558214b67/roles/05aee5278bd743c4bb8fb605feb806a9 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:28.465343 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a54b7-0734-4184-b497-a6f52d5c5a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:28.465977 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a54b7-0734-4184-b497-a6f52d5c5a9e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:28.466250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a54b7-0734-4184-b497-a6f52d5c5a9e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:28.466517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a54b7-0734-4184-b497-a6f52d5c5a9e None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:28.467310 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a54b7-0734-4184-b497-a6f52d5c5a9e None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:28.467639 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a54b7-0734-4184-b497-a6f52d5c5a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:28.469207 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a54b7-0734-4184-b497-a6f52d5c5a9e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:28.473272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067a5b7-e3a9-4b97-9d91-c2092d6e356e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:28.499226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 225/421] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:28.501451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a54b7-0734-4184-b497-a6f52d5c5a9e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:28.513782 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:28.533599 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:28.620373 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a54b7-0734-4184-b497-a6f52d5c5a9e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:28.657722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbe1dc-3e75-4e33-bbdc-a664a62663d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:28.658275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbe1dc-3e75-4e33-bbdc-a664a62663d3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:28.658545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbe1dc-3e75-4e33-bbdc-a664a62663d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:28.658809 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbe1dc-3e75-4e33-bbdc-a664a62663d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:28.669905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067a5b7-e3a9-4b97-9d91-c2092d6e356e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:28.712906 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8166cd-3580-44b4-b327-3281df18dfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:28.713547 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8166cd-3580-44b4-b327-3281df18dfba None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:28.713817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8166cd-3580-44b4-b327-3281df18dfba None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:28.714085 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8166cd-3580-44b4-b327-3281df18dfba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:28.734017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8166cd-3580-44b4-b327-3281df18dfba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-be8166cd-3580-44b4-b327-3281df18dfba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:28.738952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:28.739997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:28.740792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 197/422] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:36:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:28.747396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8166cd-3580-44b4-b327-3281df18dfba None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:28.766459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbe1dc-3e75-4e33-bbdc-a664a62663d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:28.767953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbe1dc-3e75-4e33-bbdc-a664a62663d3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:28.788409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:28.885985 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 226/423] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:36:28.891825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbe1dc-3e75-4e33-bbdc-a664a62663d3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:28.945502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:28.977392 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8166cd-3580-44b4-b327-3281df18dfba None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:29.015796 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270074d5-67a3-411b-b2bd-709d67f878ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:29.016368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270074d5-67a3-411b-b2bd-709d67f878ee None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.016634 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270074d5-67a3-411b-b2bd-709d67f878ee None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.016893 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270074d5-67a3-411b-b2bd-709d67f878ee None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.017552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270074d5-67a3-411b-b2bd-709d67f878ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:29.017848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270074d5-67a3-411b-b2bd-709d67f878ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:29.019252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270074d5-67a3-411b-b2bd-709d67f878ee None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:29.039604 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 198/424] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:29.046639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270074d5-67a3-411b-b2bd-709d67f878ee None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:29.083581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbe1dc-3e75-4e33-bbdc-a664a62663d3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:29.111142 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:29.144847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f5778-bd9a-4b95-8b8b-8f86fba3d50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:29.145790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f5778-bd9a-4b95-8b8b-8f86fba3d50f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.146066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f5778-bd9a-4b95-8b8b-8f86fba3d50f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.146333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f5778-bd9a-4b95-8b8b-8f86fba3d50f None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.147940 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f5778-bd9a-4b95-8b8b-8f86fba3d50f None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:29.149403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f5778-bd9a-4b95-8b8b-8f86fba3d50f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:29.151941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 227/425] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:28 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:29.153165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0de5e2-8322-4e2d-a2e5-d723a52435b4 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.153440 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0de5e2-8322-4e2d-a2e5-d723a52435b4 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.153705 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0de5e2-8322-4e2d-a2e5-d723a52435b4 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.159255 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 228/426] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:29.159951 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0de5e2-8322-4e2d-a2e5-d723a52435b4 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:29.169639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270074d5-67a3-411b-b2bd-709d67f878ee None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:29.205221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c2db0-81f4-4f9f-9178-7b5398ca6092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:29.205830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c2db0-81f4-4f9f-9178-7b5398ca6092 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.206167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c2db0-81f4-4f9f-9178-7b5398ca6092 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.206499 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c2db0-81f4-4f9f-9178-7b5398ca6092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.219533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:29.282104 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c2db0-81f4-4f9f-9178-7b5398ca6092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:29.283722 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c2db0-81f4-4f9f-9178-7b5398ca6092 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:29.294866 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0de5e2-8322-4e2d-a2e5-d723a52435b4 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:29.332441 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fe17d-dacd-4ba0-8277-1b5d5562ecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:29.333393 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fe17d-dacd-4ba0-8277-1b5d5562ecde None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.333730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fe17d-dacd-4ba0-8277-1b5d5562ecde None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.334057 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fe17d-dacd-4ba0-8277-1b5d5562ecde None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.335265 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fe17d-dacd-4ba0-8277-1b5d5562ecde None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:29.336799 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fe17d-dacd-4ba0-8277-1b5d5562ecde None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:29.343397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 229/427] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:29 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:29.344652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc07ac-95cd-4971-96cd-6c16151e6515 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.344989 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc07ac-95cd-4971-96cd-6c16151e6515 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.345319 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc07ac-95cd-4971-96cd-6c16151e6515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.364065 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-f0cc07ac-95cd-4971-96cd-6c16151e6515 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:29.365359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f0cc07ac-95cd-4971-96cd-6c16151e6515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:29.402848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 199/428] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:36:29.403735 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c2db0-81f4-4f9f-9178-7b5398ca6092 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:29.429398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:29.462159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 230/429] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:29.463305 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc07ac-95cd-4971-96cd-6c16151e6515 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:29.470341 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:29.564025 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c2db0-81f4-4f9f-9178-7b5398ca6092 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:29.603197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5015d66-1548-4b28-9549-1a25440b20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:29.603197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5015d66-1548-4b28-9549-1a25440b20de None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.603197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5015d66-1548-4b28-9549-1a25440b20de None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.603197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5015d66-1548-4b28-9549-1a25440b20de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.603610 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5015d66-1548-4b28-9549-1a25440b20de None admin] RBAC: Adding query filter params (group.id=c9f8172ee7854d5ab716cfd558214b67, role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:29.603610 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5015d66-1548-4b28-9549-1a25440b20de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:29.603610 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5015d66-1548-4b28-9549-1a25440b20de None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:29.609412 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 200/430] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:29 2022] GET /identity/v3/role_assignments?group.id=c9f8172ee7854d5ab716cfd558214b67&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:29.634858 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc07ac-95cd-4971-96cd-6c16151e6515 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:29.674343 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a9b29-cb89-4a76-bddd-da2db044be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:29.675424 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a9b29-cb89-4a76-bddd-da2db044be28 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.675918 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a9b29-cb89-4a76-bddd-da2db044be28 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.676387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a9b29-cb89-4a76-bddd-da2db044be28 None admin] PATH_INFO: `/v3/projects/de3741f1b798440b9e1a4f4b24dbd224` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.677992 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a9b29-cb89-4a76-bddd-da2db044be28 None admin] RBAC: Authorizing `identity:get_project(project_id=de3741f1b798440b9e1a4f4b24dbd224)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:29.679774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a9b29-cb89-4a76-bddd-da2db044be28 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:29.682204 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 231/431] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:29 2022] GET /identity/v3/projects/de3741f1b798440b9e1a4f4b24dbd224 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:29.692298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5015d66-1548-4b28-9549-1a25440b20de None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:29.812901 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5015d66-1548-4b28-9549-1a25440b20de None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:29.819947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a9b29-cb89-4a76-bddd-da2db044be28 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:29.823629 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:29.860924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7f213-fe9b-4283-a892-db525cf3d740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:29.860924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f213-fe9b-4283-a892-db525cf3d740 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.861222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f213-fe9b-4283-a892-db525cf3d740 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.861469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f213-fe9b-4283-a892-db525cf3d740 None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.862572 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7f213-fe9b-4283-a892-db525cf3d740 None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:29.864068 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7f213-fe9b-4283-a892-db525cf3d740 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:29.866266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 201/432] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:29 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:29.871885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7f213-fe9b-4283-a892-db525cf3d740 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:29.886111 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:29.940740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a9b29-cb89-4a76-bddd-da2db044be28 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:29.980011 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f65cf-9062-4069-93e0-5d320e54434f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:29.981026 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f65cf-9062-4069-93e0-5d320e54434f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:29.981453 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f65cf-9062-4069-93e0-5d320e54434f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:29.981924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f65cf-9062-4069-93e0-5d320e54434f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:29.992069 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7f213-fe9b-4283-a892-db525cf3d740 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:29.995021 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f65cf-9062-4069-93e0-5d320e54434f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-541f65cf-9062-4069-93e0-5d320e54434f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:30.002174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:30.003072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:30.008084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 232/433] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:30.011706 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f65cf-9062-4069-93e0-5d320e54434f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:30.047085 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2c8fe-0e13-4853-ab66-a993e6f6fb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:30.048358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2c8fe-0e13-4853-ab66-a993e6f6fb3e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.049176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2c8fe-0e13-4853-ab66-a993e6f6fb3e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.049927 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2c8fe-0e13-4853-ab66-a993e6f6fb3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.050905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2c8fe-0e13-4853-ab66-a993e6f6fb3e None admin] RBAC: Adding query filter params (group.id=c9f8172ee7854d5ab716cfd558214b67, role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=de3741f1b798440b9e1a4f4b24dbd224) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:30.051435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2c8fe-0e13-4853-ab66-a993e6f6fb3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:30.054377 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2c8fe-0e13-4853-ab66-a993e6f6fb3e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:30.061963 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 202/434] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:29 2022] GET /identity/v3/role_assignments?group.id=c9f8172ee7854d5ab716cfd558214b67&scope.project.id=de3741f1b798440b9e1a4f4b24dbd224&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:30.135238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f65cf-9062-4069-93e0-5d320e54434f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:30.143423 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:30.149229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df9755-2d61-41a7-bb7c-6aef004d8985 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.149872 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df9755-2d61-41a7-bb7c-6aef004d8985 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.150074 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df9755-2d61-41a7-bb7c-6aef004d8985 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.151804 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 203/435] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:36:30.157445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3b744-847b-436d-9ea4-edef05a8c96c None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.157733 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3b744-847b-436d-9ea4-edef05a8c96c None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.157978 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3b744-847b-436d-9ea4-edef05a8c96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.180403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105ef6-2e23-4072-940c-4290cc796414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:30.181386 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105ef6-2e23-4072-940c-4290cc796414 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.181869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105ef6-2e23-4072-940c-4290cc796414 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.182324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105ef6-2e23-4072-940c-4290cc796414 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.183240 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105ef6-2e23-4072-940c-4290cc796414 None admin] RBAC: Adding query filter params (name=member) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:30.183342 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-87f3b744-847b-436d-9ea4-edef05a8c96c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:30.183824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105ef6-2e23-4072-940c-4290cc796414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:30.184486 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-87f3b744-847b-436d-9ea4-edef05a8c96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:30.185628 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105ef6-2e23-4072-940c-4290cc796414 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:30.195380 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 233/436] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:36:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:30.201413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105ef6-2e23-4072-940c-4290cc796414 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:30.233488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:30.273402 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 204/437] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:30.338171 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105ef6-2e23-4072-940c-4290cc796414 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:30.372786 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8fa69-c75a-47c0-aa77-41d7109e70c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:30.373396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8fa69-c75a-47c0-aa77-41d7109e70c9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.373666 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8fa69-c75a-47c0-aa77-41d7109e70c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.373927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8fa69-c75a-47c0-aa77-41d7109e70c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.418140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:30.453119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8fa69-c75a-47c0-aa77-41d7109e70c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:30.454897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8fa69-c75a-47c0-aa77-41d7109e70c9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:30.563988 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 234/438] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:30.569949 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3b744-847b-436d-9ea4-edef05a8c96c None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:30.577394 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8fa69-c75a-47c0-aa77-41d7109e70c9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:30.666896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:30.705269 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3b744-847b-436d-9ea4-edef05a8c96c None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:30.711504 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8fa69-c75a-47c0-aa77-41d7109e70c9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:30.738975 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c813471-93c1-45c8-b071-975757062c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:30.739683 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c813471-93c1-45c8-b071-975757062c97 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.739948 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c813471-93c1-45c8-b071-975757062c97 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.740209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c813471-93c1-45c8-b071-975757062c97 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.741300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c813471-93c1-45c8-b071-975757062c97 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:30.742728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c813471-93c1-45c8-b071-975757062c97 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:30.744384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 205/439] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:30 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:30.745460 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561927b5-e1c0-4c2d-ae81-a430a7e6def6 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.745660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f32716-f6cd-4427-be7c-fe84ef8619b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:30.745768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561927b5-e1c0-4c2d-ae81-a430a7e6def6 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.745978 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561927b5-e1c0-4c2d-ae81-a430a7e6def6 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.746290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f32716-f6cd-4427-be7c-fe84ef8619b2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.746563 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f32716-f6cd-4427-be7c-fe84ef8619b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.746828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f32716-f6cd-4427-be7c-fe84ef8619b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.747366 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 206/440] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:30.751718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561927b5-e1c0-4c2d-ae81-a430a7e6def6 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:30.757156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f32716-f6cd-4427-be7c-fe84ef8619b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-07f32716-f6cd-4427-be7c-fe84ef8619b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:30.761957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:30.762902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:30.763766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 235/441] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:36:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:30.764423 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74573509-e24b-46eb-b738-4e69b8f11251 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.764486 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74573509-e24b-46eb-b738-4e69b8f11251 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.764747 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74573509-e24b-46eb-b738-4e69b8f11251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.782319 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-74573509-e24b-46eb-b738-4e69b8f11251 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:30.783523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-74573509-e24b-46eb-b738-4e69b8f11251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:30.790763 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:30.853232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 236/442] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:30.854070 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74573509-e24b-46eb-b738-4e69b8f11251 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:30.861326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561927b5-e1c0-4c2d-ae81-a430a7e6def6 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:30.904993 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9509598-04ba-42f6-a0c9-1c565d41b6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:30.905871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9509598-04ba-42f6-a0c9-1c565d41b6a3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:30.906146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9509598-04ba-42f6-a0c9-1c565d41b6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:30.906413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9509598-04ba-42f6-a0c9-1c565d41b6a3 None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:30.907554 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9509598-04ba-42f6-a0c9-1c565d41b6a3 None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:30.909222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9509598-04ba-42f6-a0c9-1c565d41b6a3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:30.910733 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 207/443] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:30 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:30.916421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9509598-04ba-42f6-a0c9-1c565d41b6a3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:30.928202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:30.973842 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74573509-e24b-46eb-b738-4e69b8f11251 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.009718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87adab2-acbb-423d-a84f-f272e7463a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.010419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87adab2-acbb-423d-a84f-f272e7463a01 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.010663 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87adab2-acbb-423d-a84f-f272e7463a01 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.010854 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87adab2-acbb-423d-a84f-f272e7463a01 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.011550 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87adab2-acbb-423d-a84f-f272e7463a01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:31.011842 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87adab2-acbb-423d-a84f-f272e7463a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:31.013224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87adab2-acbb-423d-a84f-f272e7463a01 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:31.021117 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9509598-04ba-42f6-a0c9-1c565d41b6a3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.021695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 237/444] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:31.026818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87adab2-acbb-423d-a84f-f272e7463a01 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:31.028101 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:31.062481 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db258e-8e2b-4ab1-976b-f390ab8b8d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.063093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db258e-8e2b-4ab1-976b-f390ab8b8d32 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.063399 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db258e-8e2b-4ab1-976b-f390ab8b8d32 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.063670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db258e-8e2b-4ab1-976b-f390ab8b8d32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.064454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db258e-8e2b-4ab1-976b-f390ab8b8d32 None admin] RBAC: Adding query filter params (group.id=c9f8172ee7854d5ab716cfd558214b67, role.id=91e10f3d9d934fe1be9d3642b8092d08, scope.project.id=d7a7423471544bcb8851a5df381c6144) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:31.064767 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db258e-8e2b-4ab1-976b-f390ab8b8d32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:31.066203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db258e-8e2b-4ab1-976b-f390ab8b8d32 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:31.072264 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 208/445] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:30 2022] GET /identity/v3/role_assignments?group.id=c9f8172ee7854d5ab716cfd558214b67&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=91e10f3d9d934fe1be9d3642b8092d08 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:31.120912 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db258e-8e2b-4ab1-976b-f390ab8b8d32 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:31.136774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87adab2-acbb-423d-a84f-f272e7463a01 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.158876 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:31.177446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154403d-46f3-4d39-a463-0dd5fd9edc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.178262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154403d-46f3-4d39-a463-0dd5fd9edc0d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.178537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154403d-46f3-4d39-a463-0dd5fd9edc0d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.178795 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154403d-46f3-4d39-a463-0dd5fd9edc0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.230201 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db258e-8e2b-4ab1-976b-f390ab8b8d32 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.260371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154403d-46f3-4d39-a463-0dd5fd9edc0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:31.262685 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154403d-46f3-4d39-a463-0dd5fd9edc0d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:31.268262 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edf9c4-2f57-46e6-ada3-2860ff39f7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.268905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edf9c4-2f57-46e6-ada3-2860ff39f7c4 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.269182 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edf9c4-2f57-46e6-ada3-2860ff39f7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.269454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edf9c4-2f57-46e6-ada3-2860ff39f7c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.285102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edf9c4-2f57-46e6-ada3-2860ff39f7c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-d5edf9c4-2f57-46e6-ada3-2860ff39f7c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:31.290609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:31.291570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:31.292376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 209/446] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:36:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:31.299343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edf9c4-2f57-46e6-ada3-2860ff39f7c4 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:31.339794 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:31.377950 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 238/447] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:36:31.385562 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154403d-46f3-4d39-a463-0dd5fd9edc0d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:31.406996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:31.432113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edf9c4-2f57-46e6-ada3-2860ff39f7c4 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.468054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17f8d1-4161-4345-aa8d-aa2bfd96b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.468667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17f8d1-4161-4345-aa8d-aa2bfd96b6df None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.468947 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17f8d1-4161-4345-aa8d-aa2bfd96b6df None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.469212 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17f8d1-4161-4345-aa8d-aa2bfd96b6df None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.469874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17f8d1-4161-4345-aa8d-aa2bfd96b6df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:31.470173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17f8d1-4161-4345-aa8d-aa2bfd96b6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:31.471581 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17f8d1-4161-4345-aa8d-aa2bfd96b6df None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:31.479326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 210/448] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:31.485223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17f8d1-4161-4345-aa8d-aa2bfd96b6df None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:31.490958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154403d-46f3-4d39-a463-0dd5fd9edc0d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.527214 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca24f7f-2ee4-4733-8d86-552e8a658bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.527841 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca24f7f-2ee4-4733-8d86-552e8a658bca None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.528109 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca24f7f-2ee4-4733-8d86-552e8a658bca None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.528371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca24f7f-2ee4-4733-8d86-552e8a658bca None admin] PATH_INFO: `/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.529511 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca24f7f-2ee4-4733-8d86-552e8a658bca None admin] RBAC: Authorizing `identity:get_project(project_id=a2579099690b4b9ea0cc6cb7695b8b5a)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:31.530955 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca24f7f-2ee4-4733-8d86-552e8a658bca None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:31.532616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 239/449] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:31 2022] GET /identity/v3/projects/a2579099690b4b9ea0cc6cb7695b8b5a => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:31.544383 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca24f7f-2ee4-4733-8d86-552e8a658bca None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:31.587236 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:31.597389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17f8d1-4161-4345-aa8d-aa2bfd96b6df None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.630032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3f423-21d6-4334-9827-4efac84a7b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.630635 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f423-21d6-4334-9827-4efac84a7b9f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.630915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f423-21d6-4334-9827-4efac84a7b9f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.631217 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f423-21d6-4334-9827-4efac84a7b9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.631392 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:31.645136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca24f7f-2ee4-4733-8d86-552e8a658bca None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.678061 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4fd21-438a-4ab7-948a-d5a050903dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.678682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4fd21-438a-4ab7-948a-d5a050903dd3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.678957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4fd21-438a-4ab7-948a-d5a050903dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.679288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4fd21-438a-4ab7-948a-d5a050903dd3 None admin] PATH_INFO: `/v3/groups/0a231dffa2dd4db7bcad94baf472b066` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.680414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4fd21-438a-4ab7-948a-d5a050903dd3 None admin] RBAC: Authorizing `identity:get_group(group_id=0a231dffa2dd4db7bcad94baf472b066)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:31.681856 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4fd21-438a-4ab7-948a-d5a050903dd3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:31.683492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 240/450] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:31 2022] GET /identity/v3/groups/0a231dffa2dd4db7bcad94baf472b066 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:31.689198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4fd21-438a-4ab7-948a-d5a050903dd3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:31.698269 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3f423-21d6-4334-9827-4efac84a7b9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:31.699744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3f423-21d6-4334-9827-4efac84a7b9f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:31.787490 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:31.787719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4fd21-438a-4ab7-948a-d5a050903dd3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.798540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 211/451] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:36:31.804587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3f423-21d6-4334-9827-4efac84a7b9f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:31.821839 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9404c5-d540-4bf2-b781-26c1e3733850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.822447 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9404c5-d540-4bf2-b781-26c1e3733850 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.822722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9404c5-d540-4bf2-b781-26c1e3733850 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.822991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9404c5-d540-4bf2-b781-26c1e3733850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.823743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9404c5-d540-4bf2-b781-26c1e3733850 None admin] RBAC: Adding query filter params (group.id=0a231dffa2dd4db7bcad94baf472b066, role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=a2579099690b4b9ea0cc6cb7695b8b5a) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:31.824050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9404c5-d540-4bf2-b781-26c1e3733850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:31.825496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9404c5-d540-4bf2-b781-26c1e3733850 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:31.831502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 241/452] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:31 2022] GET /identity/v3/role_assignments?group.id=0a231dffa2dd4db7bcad94baf472b066&scope.project.id=a2579099690b4b9ea0cc6cb7695b8b5a&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:31.911431 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3f423-21d6-4334-9827-4efac84a7b9f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:31.948649 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda677e5-f866-4bad-894a-03b0d54f3731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:31.949330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda677e5-f866-4bad-894a-03b0d54f3731 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:31.949621 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda677e5-f866-4bad-894a-03b0d54f3731 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:31.949954 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda677e5-f866-4bad-894a-03b0d54f3731 None admin] PATH_INFO: `/v3/projects/d7a7423471544bcb8851a5df381c6144` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:31.951265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda677e5-f866-4bad-894a-03b0d54f3731 None admin] RBAC: Authorizing `identity:get_project(project_id=d7a7423471544bcb8851a5df381c6144)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:31.952714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda677e5-f866-4bad-894a-03b0d54f3731 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:31.954707 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 212/453] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:36:31 2022] GET /identity/v3/projects/d7a7423471544bcb8851a5df381c6144 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:31.960542 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9404c5-d540-4bf2-b781-26c1e3733850 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:31.985392 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:32.063296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9404c5-d540-4bf2-b781-26c1e3733850 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:32.097811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95d25c-7ac9-46fd-a7cd-0ce387ebe95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:32.098426 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95d25c-7ac9-46fd-a7cd-0ce387ebe95f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:32.098696 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95d25c-7ac9-46fd-a7cd-0ce387ebe95f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:32.099015 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95d25c-7ac9-46fd-a7cd-0ce387ebe95f None admin] PATH_INFO: `/v3/groups/c9f8172ee7854d5ab716cfd558214b67` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:32.100198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95d25c-7ac9-46fd-a7cd-0ce387ebe95f None admin] RBAC: Authorizing `identity:get_group(group_id=c9f8172ee7854d5ab716cfd558214b67)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:32.101644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95d25c-7ac9-46fd-a7cd-0ce387ebe95f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:32.103234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 242/454] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Aug 30 11:36:31 2022] GET /identity/v3/groups/c9f8172ee7854d5ab716cfd558214b67 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:32.109979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda677e5-f866-4bad-894a-03b0d54f3731 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:32.179301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:32.230767 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda677e5-f866-4bad-894a-03b0d54f3731 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:32.266082 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f7c06-36fc-4c27-a4a6-acd3723c45c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:32.266672 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f7c06-36fc-4c27-a4a6-acd3723c45c7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:32.266941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f7c06-36fc-4c27-a4a6-acd3723c45c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:32.267230 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f7c06-36fc-4c27-a4a6-acd3723c45c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:32.267950 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f7c06-36fc-4c27-a4a6-acd3723c45c7 None admin] RBAC: Adding query filter params (group.id=c9f8172ee7854d5ab716cfd558214b67, role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=d7a7423471544bcb8851a5df381c6144) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:32.268229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f7c06-36fc-4c27-a4a6-acd3723c45c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:32.269686 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f7c06-36fc-4c27-a4a6-acd3723c45c7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:32.275580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 213/455] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Aug 30 11:36:32 2022] GET /identity/v3/role_assignments?group.id=c9f8172ee7854d5ab716cfd558214b67&scope.project.id=d7a7423471544bcb8851a5df381c6144&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:34.715315 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50870cf-4e5f-4086-8b37-f8238d60c362 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:34.716257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50870cf-4e5f-4086-8b37-f8238d60c362 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:34.716725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50870cf-4e5f-4086-8b37-f8238d60c362 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:34.718488 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 243/456] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:34.724225 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74a9a4-f821-4f51-9619-96b45bf42f11 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:34.725050 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74a9a4-f821-4f51-9619-96b45bf42f11 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:34.725526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74a9a4-f821-4f51-9619-96b45bf42f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:34.752769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-3f74a9a4-f821-4f51-9619-96b45bf42f11 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:34.754283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-3f74a9a4-f821-4f51-9619-96b45bf42f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:34.821356 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa4b64-d0ca-4138-957f-7037452642e6 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:34.821980 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa4b64-d0ca-4138-957f-7037452642e6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:34.822445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa4b64-d0ca-4138-957f-7037452642e6 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:34.824016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 244/457] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:34.829124 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b30c5-753f-4e32-a1dc-a7b9f10fc8f2 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:34.829645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b30c5-753f-4e32-a1dc-a7b9f10fc8f2 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:34.830091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b30c5-753f-4e32-a1dc-a7b9f10fc8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:34.843582 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 214/458] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:34.864158 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-d84b30c5-753f-4e32-a1dc-a7b9f10fc8f2 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:34.865681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-d84b30c5-753f-4e32-a1dc-a7b9f10fc8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:34.960072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 245/459] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:35.104848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92077f1c-f92f-4853-ae32-a06f8785963f None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.105501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92077f1c-f92f-4853-ae32-a06f8785963f None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.105957 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92077f1c-f92f-4853-ae32-a06f8785963f None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:35.107769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 215/460] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:35.112655 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc60693-f065-402c-85d6-14f6915e51de None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.112948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc60693-f065-402c-85d6-14f6915e51de None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.113221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc60693-f065-402c-85d6-14f6915e51de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:35.144126 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-acc60693-f065-402c-85d6-14f6915e51de None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:35.145309 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-acc60693-f065-402c-85d6-14f6915e51de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:35.160443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:35.173783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92077f1c-f92f-4853-ae32-a06f8785963f None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:35.254018 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 246/461] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:35.261837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:35.319739 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92077f1c-f92f-4853-ae32-a06f8785963f None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:35.364481 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd5953-8c99-4894-a661-22caf001df32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:35.365043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd5953-8c99-4894-a661-22caf001df32 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.365316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd5953-8c99-4894-a661-22caf001df32 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.365584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd5953-8c99-4894-a661-22caf001df32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:35.378553 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd5953-8c99-4894-a661-22caf001df32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:35.380068 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd5953-8c99-4894-a661-22caf001df32 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-abbd5953-8c99-4894-a661-22caf001df32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:35.396112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:35.397018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:35.398457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:35.398457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:35.398457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:35.398457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 216/462] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:35.403197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc60693-f065-402c-85d6-14f6915e51de None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:35.439516 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd5953-8c99-4894-a661-22caf001df32 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:35.520677 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc60693-f065-402c-85d6-14f6915e51de None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:35.559293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0997c95-06ae-499a-a36d-2005ccd11556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:35.560145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997c95-06ae-499a-a36d-2005ccd11556 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.560549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997c95-06ae-499a-a36d-2005ccd11556 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.560979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997c95-06ae-499a-a36d-2005ccd11556 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:35.561771 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0997c95-06ae-499a-a36d-2005ccd11556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:35.562202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0997c95-06ae-499a-a36d-2005ccd11556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:35.563771 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0997c95-06ae-499a-a36d-2005ccd11556 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:35.575793 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 247/463] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:35.581496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0997c95-06ae-499a-a36d-2005ccd11556 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:35.593741 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:35.611184 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd5953-8c99-4894-a661-22caf001df32 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:35.614415 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:35.659978 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b6fca-ec8b-4c4b-aa08-82bc734c3954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:35.660536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b6fca-ec8b-4c4b-aa08-82bc734c3954 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.660812 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b6fca-ec8b-4c4b-aa08-82bc734c3954 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.661270 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b6fca-ec8b-4c4b-aa08-82bc734c3954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:35.675144 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b6fca-ec8b-4c4b-aa08-82bc734c3954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:35.676612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b6fca-ec8b-4c4b-aa08-82bc734c3954 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:35.695500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0997c95-06ae-499a-a36d-2005ccd11556 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-949b6fca-ec8b-4c4b-aa08-82bc734c3954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:35.697514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:35.698363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:35.699233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:35.699233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:35.699233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:35.699233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 217/464] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:35.699808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b6fca-ec8b-4c4b-aa08-82bc734c3954 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:35.735889 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb4b8-2f35-49dc-a472-2f698ccc4b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:35.736561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb4b8-2f35-49dc-a472-2f698ccc4b55 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.736846 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb4b8-2f35-49dc-a472-2f698ccc4b55 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.737061 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb4b8-2f35-49dc-a472-2f698ccc4b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:35.748405 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb4b8-2f35-49dc-a472-2f698ccc4b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:35.750052 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb4b8-2f35-49dc-a472-2f698ccc4b55 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-634bb4b8-2f35-49dc-a472-2f698ccc4b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:35.765971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:35.766900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:35.768173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:35.768173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:35.768173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:35.768173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 248/465] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:35.768391 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5061ec5-c31c-44ff-a74c-31ca13dd7451 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.768854 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5061ec5-c31c-44ff-a74c-31ca13dd7451 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.768854 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5061ec5-c31c-44ff-a74c-31ca13dd7451 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:35.770148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 249/466] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:35.770730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5061ec5-c31c-44ff-a74c-31ca13dd7451 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:35.813296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b6fca-ec8b-4c4b-aa08-82bc734c3954 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:35.837302 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:35.849802 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b54f-768c-4c9d-881f-99f42ddc9838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:35.850740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b54f-768c-4c9d-881f-99f42ddc9838 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.850740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b54f-768c-4c9d-881f-99f42ddc9838 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.850934 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b54f-768c-4c9d-881f-99f42ddc9838 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:35.851613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8b54f-768c-4c9d-881f-99f42ddc9838 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:35.853055 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8b54f-768c-4c9d-881f-99f42ddc9838 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:35.876172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 218/467] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:36:35 2022] POST /identity/v3/users => generated 246 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:36:35.877199 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b54f-768c-4c9d-881f-99f42ddc9838 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:35.900404 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5061ec5-c31c-44ff-a74c-31ca13dd7451 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:35.901731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:35.951283 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b04056-897f-429f-8ad3-e5e2b0cdd2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:35.951848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b04056-897f-429f-8ad3-e5e2b0cdd2d1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.952124 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b04056-897f-429f-8ad3-e5e2b0cdd2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.952394 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b04056-897f-429f-8ad3-e5e2b0cdd2d1 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:35.953246 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b04056-897f-429f-8ad3-e5e2b0cdd2d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:35.953551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b04056-897f-429f-8ad3-e5e2b0cdd2d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:35.956058 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b04056-897f-429f-8ad3-e5e2b0cdd2d1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:35.973847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 250/468] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:35.975156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c230ce-63f4-4bd6-9c86-ca3f5f8f5173 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:35.975444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c230ce-63f4-4bd6-9c86-ca3f5f8f5173 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:35.975716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c230ce-63f4-4bd6-9c86-ca3f5f8f5173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:36.000379 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-28c230ce-63f4-4bd6-9c86-ca3f5f8f5173 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:36.001896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-28c230ce-63f4-4bd6-9c86-ca3f5f8f5173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:36.013894 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b54f-768c-4c9d-881f-99f42ddc9838 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:36.052096 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db19a4-4404-45ff-8527-becd2b53e22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:36.052711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db19a4-4404-45ff-8527-becd2b53e22a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:36.052984 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db19a4-4404-45ff-8527-becd2b53e22a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:36.053256 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db19a4-4404-45ff-8527-becd2b53e22a None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:36.053903 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db19a4-4404-45ff-8527-becd2b53e22a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:36.054201 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db19a4-4404-45ff-8527-becd2b53e22a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:36.055655 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db19a4-4404-45ff-8527-becd2b53e22a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:36.068194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 219/469] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:36.068980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db19a4-4404-45ff-8527-becd2b53e22a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:36.084253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 251/470] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:36.085202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c230ce-63f4-4bd6-9c86-ca3f5f8f5173 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:36.150652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:36.165933 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:36.200715 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db19a4-4404-45ff-8527-becd2b53e22a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:36.218740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c230ce-63f4-4bd6-9c86-ca3f5f8f5173 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:36.240501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a3519-6163-4718-9735-dcdc0362042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:36.241109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a3519-6163-4718-9735-dcdc0362042c None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:36.241379 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a3519-6163-4718-9735-dcdc0362042c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:36.241644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a3519-6163-4718-9735-dcdc0362042c None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:36.242281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a3519-6163-4718-9735-dcdc0362042c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:36.243793 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a3519-6163-4718-9735-dcdc0362042c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:36.264512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531ff67-3131-4ac8-acc0-7192b5a944ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:36.265193 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ff67-3131-4ac8-acc0-7192b5a944ce None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:36.265474 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ff67-3131-4ac8-acc0-7192b5a944ce None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:36.265721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ff67-3131-4ac8-acc0-7192b5a944ce None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:36.266371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531ff67-3131-4ac8-acc0-7192b5a944ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:36.268149 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531ff67-3131-4ac8-acc0-7192b5a944ce None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:36.273113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 220/471] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:36:36 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:36:36.297081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 252/472] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:36:36 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:36:36.372449 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6345e-ea3c-44a7-98cf-71af0303fd1e None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:36.372972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6345e-ea3c-44a7-98cf-71af0303fd1e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:36.373370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6345e-ea3c-44a7-98cf-71af0303fd1e None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:36.375473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 221/473] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:36.381165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d06230-81b7-4a92-a5cc-3f1772868ace None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:36.381655 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d06230-81b7-4a92-a5cc-3f1772868ace None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:36.381985 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d06230-81b7-4a92-a5cc-3f1772868ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:36.387070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6345e-ea3c-44a7-98cf-71af0303fd1e None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:36.406293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-b3d06230-81b7-4a92-a5cc-3f1772868ace None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:36.410332 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-b3d06230-81b7-4a92-a5cc-3f1772868ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:36.518106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 253/474] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:36.543136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6345e-ea3c-44a7-98cf-71af0303fd1e None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:36.570777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:36.579716 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca716b1-d6cc-4707-ab46-bd92de0e6083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:36.580355 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca716b1-d6cc-4707-ab46-bd92de0e6083 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:36.580633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca716b1-d6cc-4707-ab46-bd92de0e6083 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:36.580907 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca716b1-d6cc-4707-ab46-bd92de0e6083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:36.592875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca716b1-d6cc-4707-ab46-bd92de0e6083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:36.594355 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca716b1-d6cc-4707-ab46-bd92de0e6083 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-aca716b1-d6cc-4707-ab46-bd92de0e6083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:36.609587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:36.610558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:36.611476 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:36.611476 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:36.611476 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:36.611476 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 222/475] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:36.616526 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d06230-81b7-4a92-a5cc-3f1772868ace None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:36.652265 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:36.728292 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d06230-81b7-4a92-a5cc-3f1772868ace None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:36.763140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c4e0-6a12-4291-a503-ba26f2fb2413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:36.763780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c4e0-6a12-4291-a503-ba26f2fb2413 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:36.764017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c4e0-6a12-4291-a503-ba26f2fb2413 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:36.764283 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c4e0-6a12-4291-a503-ba26f2fb2413 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:36.764924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72c4e0-6a12-4291-a503-ba26f2fb2413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:36.765221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72c4e0-6a12-4291-a503-ba26f2fb2413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:36.766600 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72c4e0-6a12-4291-a503-ba26f2fb2413 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:36.778666 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 254/476] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:36.783943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca716b1-d6cc-4707-ab46-bd92de0e6083 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:36.831509 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c4e0-6a12-4291-a503-ba26f2fb2413 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:36.975837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c4e0-6a12-4291-a503-ba26f2fb2413 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:36.983455 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:37.011472 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:37.012396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:37.012675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:37.012942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:37.013614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:37.014988 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:37.024431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.sql.core [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 11:36:37.024431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 11:36:37.024431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [parameters: {'id': '4f00b3a8763c4fea80b2f4872fb707c6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 11:36:37.024431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87980) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application conn.query(q) Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application result.read() Aug 30 11:36:37.043051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 11:36:37.043952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application next(self.gen) Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application yield resource Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application next(self.gen) Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise self.value Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application session.commit() Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 11:36:37.044862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self.session.flush() Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application compat.raise_( Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 11:36:37.045719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 11:36:37.046586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application conn.query(q) Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application result.read() Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 11:36:37.047396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [parameters: {'id': '4f00b3a8763c4fea80b2f4872fb707c6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:37.048182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 11:36:37.049038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:37.049875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 255/477] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Aug 30 11:36:36 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:37.049938 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca716b1-d6cc-4707-ab46-bd92de0e6083 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:37.051745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:37.158516 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e688f28-0d92-4248-9642-608fd1ebdeac None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:37.176735 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cef2f-86ad-45b8-9810-7ea0b0302215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:37.177367 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cef2f-86ad-45b8-9810-7ea0b0302215 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:37.177651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cef2f-86ad-45b8-9810-7ea0b0302215 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:37.177922 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cef2f-86ad-45b8-9810-7ea0b0302215 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:37.178584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cef2f-86ad-45b8-9810-7ea0b0302215 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:37.180055 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cef2f-86ad-45b8-9810-7ea0b0302215 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:37.200203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60c937-ddc6-4362-a771-fdb46e6c3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:37.200849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60c937-ddc6-4362-a771-fdb46e6c3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:37.201116 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60c937-ddc6-4362-a771-fdb46e6c3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:37.201374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60c937-ddc6-4362-a771-fdb46e6c3ec7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:37.212819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60c937-ddc6-4362-a771-fdb46e6c3ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-5c60c937-ddc6-4362-a771-fdb46e6c3ec7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:37.218767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:37.221010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:37.223675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 256/478] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:36:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:37.228330 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60c937-ddc6-4362-a771-fdb46e6c3ec7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:37.250804 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 223/479] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:36:36 2022] POST /identity/v3/users => generated 249 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:36:37.255568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:37.346073 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60c937-ddc6-4362-a771-fdb46e6c3ec7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:37.383000 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af59cc-f4d3-456b-9049-491bbd4b3055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:37.383662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af59cc-f4d3-456b-9049-491bbd4b3055 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:37.383935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af59cc-f4d3-456b-9049-491bbd4b3055 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:37.384200 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af59cc-f4d3-456b-9049-491bbd4b3055 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:37.384916 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af59cc-f4d3-456b-9049-491bbd4b3055 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:37.385213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af59cc-f4d3-456b-9049-491bbd4b3055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:37.386671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af59cc-f4d3-456b-9049-491bbd4b3055 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:37.396342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 257/480] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:38.440155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186425f6-7c8d-43ba-8ad6-53330ac071ed None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:38.440592 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186425f6-7c8d-43ba-8ad6-53330ac071ed None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:38.440694 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186425f6-7c8d-43ba-8ad6-53330ac071ed None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:38.442342 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 224/481] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:38.447937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20767e-1495-41a7-82f8-892c99b39dc4 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:38.448341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20767e-1495-41a7-82f8-892c99b39dc4 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:38.448524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20767e-1495-41a7-82f8-892c99b39dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:38.467415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-ee20767e-1495-41a7-82f8-892c99b39dc4 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:38.468601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ee20767e-1495-41a7-82f8-892c99b39dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:38.547844 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 258/482] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:38.719783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427ca0e-5b37-42bd-bbb0-fb541b9d6b58 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:38.720078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427ca0e-5b37-42bd-bbb0-fb541b9d6b58 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:38.720255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427ca0e-5b37-42bd-bbb0-fb541b9d6b58 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:38.721880 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 225/483] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:38.726843 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f89689-82f5-43c4-8a51-d07361de3b56 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:38.727146 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f89689-82f5-43c4-8a51-d07361de3b56 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:38.727426 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f89689-82f5-43c4-8a51-d07361de3b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:38.750432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-f9f89689-82f5-43c4-8a51-d07361de3b56 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:38.751681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f9f89689-82f5-43c4-8a51-d07361de3b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:38.764384 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:38.829484 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826d926-e665-47f4-973b-969fabace4f5 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:38.829792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826d926-e665-47f4-973b-969fabace4f5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:38.830031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826d926-e665-47f4-973b-969fabace4f5 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:38.831661 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 226/484] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:36:38.836359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e923f-de74-48c6-8ca5-b493822576a2 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:38.836639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e923f-de74-48c6-8ca5-b493822576a2 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:38.836911 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e923f-de74-48c6-8ca5-b493822576a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:38.841691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 259/485] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:38.863292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-765e923f-de74-48c6-8ca5-b493822576a2 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:38.864530 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-765e923f-de74-48c6-8ca5-b493822576a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:38.908848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f89689-82f5-43c4-8a51-d07361de3b56 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:38.917359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:38.990893 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 227/486] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:39.053840 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f89689-82f5-43c4-8a51-d07361de3b56 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:39.098585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8f88-960b-4a1a-8a6e-aed4c6e1cfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:39.099601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8f88-960b-4a1a-8a6e-aed4c6e1cfec None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.100068 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8f88-960b-4a1a-8a6e-aed4c6e1cfec None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.100520 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8f88-960b-4a1a-8a6e-aed4c6e1cfec None admin] PATH_INFO: `/v3/roles/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.115920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a8f88-960b-4a1a-8a6e-aed4c6e1cfec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-743a8f88-960b-4a1a-8a6e-aed4c6e1cfec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:39.121187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:39.122041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:39.122827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:39.125437 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 260/487] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:39.132200 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e923f-de74-48c6-8ca5-b493822576a2 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:39.135401 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8f88-960b-4a1a-8a6e-aed4c6e1cfec None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:39.141226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:39.292861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e923f-de74-48c6-8ca5-b493822576a2 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:39.307081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8f88-960b-4a1a-8a6e-aed4c6e1cfec None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:39.337041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed120bf-7131-4701-bd75-34ca03189ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:39.337877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed120bf-7131-4701-bd75-34ca03189ebc None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.338532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed120bf-7131-4701-bd75-34ca03189ebc None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.338804 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed120bf-7131-4701-bd75-34ca03189ebc None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.339767 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed120bf-7131-4701-bd75-34ca03189ebc None admin] RBAC: Adding query filter params (name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:39.340063 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed120bf-7131-4701-bd75-34ca03189ebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:39.341436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed120bf-7131-4701-bd75-34ca03189ebc None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:39.343136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd2276-da2f-464a-b539-90c6ba88020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:39.343898 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd2276-da2f-464a-b539-90c6ba88020f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.344237 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd2276-da2f-464a-b539-90c6ba88020f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.344587 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd2276-da2f-464a-b539-90c6ba88020f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.351536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 228/488] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:39.353200 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed120bf-7131-4701-bd75-34ca03189ebc None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:39.355991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd2276-da2f-464a-b539-90c6ba88020f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-f3cd2276-da2f-464a-b539-90c6ba88020f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:39.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:39.361956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:39.362819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 261/489] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:39.363514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd2276-da2f-464a-b539-90c6ba88020f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:39.390427 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:39.443602 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:39.487048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd2276-da2f-464a-b539-90c6ba88020f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:39.492023 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed120bf-7131-4701-bd75-34ca03189ebc None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:39.528860 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e4266-54c6-4398-b45c-fe88c916f8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:39.529073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff23b3-f60c-4bca-b09d-9bb04693aeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:39.529728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff23b3-f60c-4bca-b09d-9bb04693aeda None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.530011 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e4266-54c6-4398-b45c-fe88c916f8dc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.530175 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff23b3-f60c-4bca-b09d-9bb04693aeda None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.530537 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff23b3-f60c-4bca-b09d-9bb04693aeda None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.530689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e4266-54c6-4398-b45c-fe88c916f8dc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.531077 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e4266-54c6-4398-b45c-fe88c916f8dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.542124 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff23b3-f60c-4bca-b09d-9bb04693aeda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:39.543496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e4266-54c6-4398-b45c-fe88c916f8dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:39.545101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e4266-54c6-4398-b45c-fe88c916f8dc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-a7ff23b3-f60c-4bca-b09d-9bb04693aeda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:39.548161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:39.549036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:39.549809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:39.554474 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 229/490] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:39.555534 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff23b3-f60c-4bca-b09d-9bb04693aeda None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-ed7e4266-54c6-4398-b45c-fe88c916f8dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:39.565152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:39.566086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:39.566910 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:39.566910 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:39.566910 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:39.568646 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 262/491] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:39.569264 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e4266-54c6-4398-b45c-fe88c916f8dc None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:39.654990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:39.668273 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff23b3-f60c-4bca-b09d-9bb04693aeda None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:39.687986 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e4266-54c6-4398-b45c-fe88c916f8dc None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:39.701743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:39.705268 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a2548-7134-494a-a7f4-ac1d98a645b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:39.706070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a2548-7134-494a-a7f4-ac1d98a645b6 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.706384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a2548-7134-494a-a7f4-ac1d98a645b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.706697 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a2548-7134-494a-a7f4-ac1d98a645b6 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.707386 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a2548-7134-494a-a7f4-ac1d98a645b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:39.707704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a2548-7134-494a-a7f4-ac1d98a645b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:39.709069 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a2548-7134-494a-a7f4-ac1d98a645b6 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:39.717724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 230/492] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:39.718849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a2548-7134-494a-a7f4-ac1d98a645b6 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:39.728657 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42282304-2068-4fb3-889e-ce8b30445736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:39.729274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42282304-2068-4fb3-889e-ce8b30445736 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.729551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42282304-2068-4fb3-889e-ce8b30445736 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.729819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42282304-2068-4fb3-889e-ce8b30445736 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.730481 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42282304-2068-4fb3-889e-ce8b30445736 None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:39.730778 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42282304-2068-4fb3-889e-ce8b30445736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:39.732205 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42282304-2068-4fb3-889e-ce8b30445736 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:39.745107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 263/493] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:39.746451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae95786-25c9-42de-96cb-8e7212a78730 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.746741 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae95786-25c9-42de-96cb-8e7212a78730 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.747014 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae95786-25c9-42de-96cb-8e7212a78730 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.751268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 264/494] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:39.751268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae95786-25c9-42de-96cb-8e7212a78730 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:39.830184 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a2548-7134-494a-a7f4-ac1d98a645b6 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:39.856191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae95786-25c9-42de-96cb-8e7212a78730 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:39.863572 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ee5c2-84ce-43d1-9e8c-07f81ff6ac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:39.864181 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ee5c2-84ce-43d1-9e8c-07f81ff6ac3f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.864552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ee5c2-84ce-43d1-9e8c-07f81ff6ac3f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.864822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ee5c2-84ce-43d1-9e8c-07f81ff6ac3f None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.865469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ee5c2-84ce-43d1-9e8c-07f81ff6ac3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:39.865766 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ee5c2-84ce-43d1-9e8c-07f81ff6ac3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:39.867205 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ee5c2-84ce-43d1-9e8c-07f81ff6ac3f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:39.879099 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 231/495] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:39.879939 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ee5c2-84ce-43d1-9e8c-07f81ff6ac3f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:39.890010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462724d-392f-40b4-8c2d-92b2a190eb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:39.890653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462724d-392f-40b4-8c2d-92b2a190eb03 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.890927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462724d-392f-40b4-8c2d-92b2a190eb03 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.891222 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462724d-392f-40b4-8c2d-92b2a190eb03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.891835 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:39.903450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462724d-392f-40b4-8c2d-92b2a190eb03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:39.904958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462724d-392f-40b4-8c2d-92b2a190eb03 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-a462724d-392f-40b4-8c2d-92b2a190eb03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:39.919513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:39.920414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:39.921288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:39.921288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:39.921288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:39.921288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 265/496] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:39.921894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0be743-c788-4dcb-91f3-30a550784a45 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:39.922170 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0be743-c788-4dcb-91f3-30a550784a45 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:39.922437 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0be743-c788-4dcb-91f3-30a550784a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:39.940783 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-7a0be743-c788-4dcb-91f3-30a550784a45 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:39.941954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-7a0be743-c788-4dcb-91f3-30a550784a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:39.965018 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:39.979769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ee5c2-84ce-43d1-9e8c-07f81ff6ac3f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.008569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 266/497] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:40.009853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b391403-e20f-4333-86a4-698fcdfa309c None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.010131 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b391403-e20f-4333-86a4-698fcdfa309c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.010398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b391403-e20f-4333-86a4-698fcdfa309c None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.011654 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 267/498] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:40.012331 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b391403-e20f-4333-86a4-698fcdfa309c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.017358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25320b-a3af-4f71-8c97-a3b7250124bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.017989 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25320b-a3af-4f71-8c97-a3b7250124bc None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.018262 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25320b-a3af-4f71-8c97-a3b7250124bc None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.018528 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25320b-a3af-4f71-8c97-a3b7250124bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.029391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25320b-a3af-4f71-8c97-a3b7250124bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:40.030935 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25320b-a3af-4f71-8c97-a3b7250124bc None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-de25320b-a3af-4f71-8c97-a3b7250124bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:40.046214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:40.047076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:40.048028 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:40.048028 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:40.048028 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:40.048028 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 232/499] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:40.048236 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25320b-a3af-4f71-8c97-a3b7250124bc None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.111570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:40.128910 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b391403-e20f-4333-86a4-698fcdfa309c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.152298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25320b-a3af-4f71-8c97-a3b7250124bc None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.163822 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d494f92-e4b1-4385-b649-fa290d36dfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.164452 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d494f92-e4b1-4385-b649-fa290d36dfb1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.164718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d494f92-e4b1-4385-b649-fa290d36dfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.164988 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d494f92-e4b1-4385-b649-fa290d36dfb1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.172010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d494f92-e4b1-4385-b649-fa290d36dfb1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:40.173470 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d494f92-e4b1-4385-b649-fa290d36dfb1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:40.185808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636d26d-60a3-45f4-91e7-0da3280471bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-0d494f92-e4b1-4385-b649-fa290d36dfb1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:40.186203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:40.187029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:36:40.187942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:40.188513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636d26d-60a3-45f4-91e7-0da3280471bf None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.188513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636d26d-60a3-45f4-91e7-0da3280471bf None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.188513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636d26d-60a3-45f4-91e7-0da3280471bf None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.188689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 268/500] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:40.188753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636d26d-60a3-45f4-91e7-0da3280471bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:40.188753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636d26d-60a3-45f4-91e7-0da3280471bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:40.188875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e2da4-4053-48c9-9ee6-3c14d3ee668f None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.188875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e2da4-4053-48c9-9ee6-3c14d3ee668f None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.189834 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e2da4-4053-48c9-9ee6-3c14d3ee668f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.189931 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636d26d-60a3-45f4-91e7-0da3280471bf None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:40.202469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 233/501] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:40.203308 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636d26d-60a3-45f4-91e7-0da3280471bf None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.213008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-dc1e2da4-4053-48c9-9ee6-3c14d3ee668f None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:40.214534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-dc1e2da4-4053-48c9-9ee6-3c14d3ee668f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:40.229401 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:40.286708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 269/502] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:40.287567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e2da4-4053-48c9-9ee6-3c14d3ee668f None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.310660 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636d26d-60a3-45f4-91e7-0da3280471bf None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.334259 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:40.347209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277386f2-4620-41c1-9265-d567999da408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.347825 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277386f2-4620-41c1-9265-d567999da408 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.348104 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277386f2-4620-41c1-9265-d567999da408 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.348378 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277386f2-4620-41c1-9265-d567999da408 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.349101 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277386f2-4620-41c1-9265-d567999da408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:40.349405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277386f2-4620-41c1-9265-d567999da408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:40.350775 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277386f2-4620-41c1-9265-d567999da408 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:40.365417 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 234/503] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:40.366851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277386f2-4620-41c1-9265-d567999da408 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.390878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e2da4-4053-48c9-9ee6-3c14d3ee668f None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.426848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b27860-cec1-4428-8509-ceedc7c3c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.427501 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b27860-cec1-4428-8509-ceedc7c3c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.427781 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b27860-cec1-4428-8509-ceedc7c3c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.428047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b27860-cec1-4428-8509-ceedc7c3c6c1 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.428770 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b27860-cec1-4428-8509-ceedc7c3c6c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:40.429076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b27860-cec1-4428-8509-ceedc7c3c6c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:40.430534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b27860-cec1-4428-8509-ceedc7c3c6c1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:40.467482 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 270/504] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:40.468485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b27860-cec1-4428-8509-ceedc7c3c6c1 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.478138 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:40.485378 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277386f2-4620-41c1-9265-d567999da408 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.520140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81759565-767d-404c-8ae9-8f2aacfd3c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.520855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81759565-767d-404c-8ae9-8f2aacfd3c5f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.521101 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81759565-767d-404c-8ae9-8f2aacfd3c5f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.521374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81759565-767d-404c-8ae9-8f2aacfd3c5f None admin] PATH_INFO: `/v3/users/glance` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.529179 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81759565-767d-404c-8ae9-8f2aacfd3c5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:40.530632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81759565-767d-404c-8ae9-8f2aacfd3c5f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-81759565-767d-404c-8ae9-8f2aacfd3c5f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:40.543221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:40.544140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:40.545008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 235/505] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:40.545557 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81759565-767d-404c-8ae9-8f2aacfd3c5f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.584820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b27860-cec1-4428-8509-ceedc7c3c6c1 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.592725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:40.621212 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6848d9-f9d5-4408-8b99-d5a3ab10c087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.621682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6848d9-f9d5-4408-8b99-d5a3ab10c087 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.621946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6848d9-f9d5-4408-8b99-d5a3ab10c087 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.622200 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6848d9-f9d5-4408-8b99-d5a3ab10c087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.632515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6848d9-f9d5-4408-8b99-d5a3ab10c087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:40.634047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6848d9-f9d5-4408-8b99-d5a3ab10c087 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-0a6848d9-f9d5-4408-8b99-d5a3ab10c087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:40.648701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:40.649585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:40.650410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:40.650410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:40.650410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:40.650410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 271/506] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:40.650631 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6848d9-f9d5-4408-8b99-d5a3ab10c087 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.651376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81759565-767d-404c-8ae9-8f2aacfd3c5f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.687044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49718a83-47cb-4084-95ca-ad4cc3bbbc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.687699 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49718a83-47cb-4084-95ca-ad4cc3bbbc5d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.687973 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49718a83-47cb-4084-95ca-ad4cc3bbbc5d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.688236 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49718a83-47cb-4084-95ca-ad4cc3bbbc5d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.695579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49718a83-47cb-4084-95ca-ad4cc3bbbc5d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:40.697043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49718a83-47cb-4084-95ca-ad4cc3bbbc5d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:40.708416 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-49718a83-47cb-4084-95ca-ad4cc3bbbc5d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:40.708535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:40.709413 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 236/507] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:40.710406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49718a83-47cb-4084-95ca-ad4cc3bbbc5d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.754419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6848d9-f9d5-4408-8b99-d5a3ab10c087 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.789515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d0eba-ecb8-40c2-8ef1-9049c7cef4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.790132 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d0eba-ecb8-40c2-8ef1-9049c7cef4d3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.790388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d0eba-ecb8-40c2-8ef1-9049c7cef4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.790655 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d0eba-ecb8-40c2-8ef1-9049c7cef4d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.811497 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49718a83-47cb-4084-95ca-ad4cc3bbbc5d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:40.812977 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:40.847323 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147a289-82dd-4455-bbb7-8b63a36e2d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:40.847941 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147a289-82dd-4455-bbb7-8b63a36e2d7b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:40.848216 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147a289-82dd-4455-bbb7-8b63a36e2d7b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:40.848482 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147a289-82dd-4455-bbb7-8b63a36e2d7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:40.856976 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d0eba-ecb8-40c2-8ef1-9049c7cef4d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:40.858497 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d0eba-ecb8-40c2-8ef1-9049c7cef4d3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:40.860712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147a289-82dd-4455-bbb7-8b63a36e2d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-d147a289-82dd-4455-bbb7-8b63a36e2d7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:40.867524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:40.868448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:40.869251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 237/508] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:40.869920 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147a289-82dd-4455-bbb7-8b63a36e2d7b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.920039 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:40.965064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 272/509] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:40.965923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d0eba-ecb8-40c2-8ef1-9049c7cef4d3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:40.972918 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147a289-82dd-4455-bbb7-8b63a36e2d7b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.005550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935ab32-8e83-488a-8916-a1b1f3a216e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.006138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ab32-8e83-488a-8916-a1b1f3a216e2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.006475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ab32-8e83-488a-8916-a1b1f3a216e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.006742 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ab32-8e83-488a-8916-a1b1f3a216e2 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.007482 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935ab32-8e83-488a-8916-a1b1f3a216e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:41.007832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935ab32-8e83-488a-8916-a1b1f3a216e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.009265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935ab32-8e83-488a-8916-a1b1f3a216e2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.044203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 238/510] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:41.044998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935ab32-8e83-488a-8916-a1b1f3a216e2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.065038 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:41.072105 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d0eba-ecb8-40c2-8ef1-9049c7cef4d3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.105823 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c85a91-e97e-4bbb-97e7-7708b259a5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.106423 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c85a91-e97e-4bbb-97e7-7708b259a5b9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.106688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c85a91-e97e-4bbb-97e7-7708b259a5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.106950 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c85a91-e97e-4bbb-97e7-7708b259a5b9 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.107604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c85a91-e97e-4bbb-97e7-7708b259a5b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:41.107898 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c85a91-e97e-4bbb-97e7-7708b259a5b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.109277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c85a91-e97e-4bbb-97e7-7708b259a5b9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.119979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 273/511] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:41.120766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c85a91-e97e-4bbb-97e7-7708b259a5b9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.141320 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:41.145295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935ab32-8e83-488a-8916-a1b1f3a216e2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.178323 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d4e20-6ab8-46eb-9e1a-f2a89872d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.178871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d4e20-6ab8-46eb-9e1a-f2a89872d725 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.179145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d4e20-6ab8-46eb-9e1a-f2a89872d725 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.179434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d4e20-6ab8-46eb-9e1a-f2a89872d725 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.180173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d4e20-6ab8-46eb-9e1a-f2a89872d725 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:41.180451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d4e20-6ab8-46eb-9e1a-f2a89872d725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.181920 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d4e20-6ab8-46eb-9e1a-f2a89872d725 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.198733 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 239/512] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Aug 30 11:36:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:41.199562 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d4e20-6ab8-46eb-9e1a-f2a89872d725 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.220121 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c85a91-e97e-4bbb-97e7-7708b259a5b9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.253899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494b693-52fb-4d19-b3a2-cb35870d73c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.254510 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494b693-52fb-4d19-b3a2-cb35870d73c4 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.254814 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494b693-52fb-4d19-b3a2-cb35870d73c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.255078 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494b693-52fb-4d19-b3a2-cb35870d73c4 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.255760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494b693-52fb-4d19-b3a2-cb35870d73c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:41.256054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494b693-52fb-4d19-b3a2-cb35870d73c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.257432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494b693-52fb-4d19-b3a2-cb35870d73c4 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.265926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 274/513] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:41.266869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494b693-52fb-4d19-b3a2-cb35870d73c4 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.281357 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:41.304623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d4e20-6ab8-46eb-9e1a-f2a89872d725 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.339298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ef4f9-f00e-4c81-99dd-3f7eb2e357bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.339921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ef4f9-f00e-4c81-99dd-3f7eb2e357bc None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.340197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ef4f9-f00e-4c81-99dd-3f7eb2e357bc None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.340465 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ef4f9-f00e-4c81-99dd-3f7eb2e357bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.350269 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ef4f9-f00e-4c81-99dd-3f7eb2e357bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.351756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ef4f9-f00e-4c81-99dd-3f7eb2e357bc None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.352466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-258ef4f9-f00e-4c81-99dd-3f7eb2e357bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:41.367218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:41.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:41.369150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:41.369150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:41.369150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:41.369150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 240/514] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:41.369150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ef4f9-f00e-4c81-99dd-3f7eb2e357bc None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.375640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494b693-52fb-4d19-b3a2-cb35870d73c4 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.408494 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c4049-b47a-42d0-ac91-ed10183c9feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.409097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c4049-b47a-42d0-ac91-ed10183c9feb None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.409365 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c4049-b47a-42d0-ac91-ed10183c9feb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.409679 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c4049-b47a-42d0-ac91-ed10183c9feb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.466880 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ef4f9-f00e-4c81-99dd-3f7eb2e357bc None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.476518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c4049-b47a-42d0-ac91-ed10183c9feb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.477961 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c4049-b47a-42d0-ac91-ed10183c9feb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.494545 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:41.501019 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c967a-700a-4c93-ae47-0aca44bef463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.501655 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c967a-700a-4c93-ae47-0aca44bef463 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.501933 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c967a-700a-4c93-ae47-0aca44bef463 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.502202 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c967a-700a-4c93-ae47-0aca44bef463 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.502839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c967a-700a-4c93-ae47-0aca44bef463 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.504287 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c967a-700a-4c93-ae47-0aca44bef463 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.524850 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 241/515] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:36:41 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:36:41.525650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c967a-700a-4c93-ae47-0aca44bef463 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.572922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:41.603001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 275/516] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:41.604153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c4049-b47a-42d0-ac91-ed10183c9feb None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.639475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c967a-700a-4c93-ae47-0aca44bef463 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.679771 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d387e9a-9b84-4071-be5d-d4d919f42605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.680371 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d387e9a-9b84-4071-be5d-d4d919f42605 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.680659 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d387e9a-9b84-4071-be5d-d4d919f42605 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.680922 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d387e9a-9b84-4071-be5d-d4d919f42605 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.726079 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c4049-b47a-42d0-ac91-ed10183c9feb None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.742720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:41.758510 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d387e9a-9b84-4071-be5d-d4d919f42605 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.760003 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d387e9a-9b84-4071-be5d-d4d919f42605 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.763686 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ff811-4ecf-4da6-9e53-58b32254a4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.764377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ff811-4ecf-4da6-9e53-58b32254a4fc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.764654 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ff811-4ecf-4da6-9e53-58b32254a4fc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.764921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ff811-4ecf-4da6-9e53-58b32254a4fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.775312 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ff811-4ecf-4da6-9e53-58b32254a4fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.776770 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ff811-4ecf-4da6-9e53-58b32254a4fc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-bb5ff811-4ecf-4da6-9e53-58b32254a4fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:41.791551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:41.792489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:41.793317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:41.793317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:41.793317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:41.793317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 276/517] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:41.793535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ff811-4ecf-4da6-9e53-58b32254a4fc None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.827972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:41.872807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 242/518] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:41.872807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d387e9a-9b84-4071-be5d-d4d919f42605 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.899553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ff811-4ecf-4da6-9e53-58b32254a4fc None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:41.936972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e71f7-13a0-4b44-a95a-8033318a1dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:41.937586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e71f7-13a0-4b44-a95a-8033318a1dbc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:41.937863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e71f7-13a0-4b44-a95a-8033318a1dbc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:41.938134 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e71f7-13a0-4b44-a95a-8033318a1dbc None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:41.938772 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e71f7-13a0-4b44-a95a-8033318a1dbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:41.939071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e71f7-13a0-4b44-a95a-8033318a1dbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:41.940700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e71f7-13a0-4b44-a95a-8033318a1dbc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:41.959275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 277/519] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:41.959275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e71f7-13a0-4b44-a95a-8033318a1dbc None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:41.961238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:41.979162 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d387e9a-9b84-4071-be5d-d4d919f42605 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.015777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5b70f-ca90-45c7-9f5f-e536a4436a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.016477 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5b70f-ca90-45c7-9f5f-e536a4436a4c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.016740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5b70f-ca90-45c7-9f5f-e536a4436a4c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.017007 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5b70f-ca90-45c7-9f5f-e536a4436a4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.027678 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5b70f-ca90-45c7-9f5f-e536a4436a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.029156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5b70f-ca90-45c7-9f5f-e536a4436a4c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-b8e5b70f-ca90-45c7-9f5f-e536a4436a4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:42.044113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:42.045030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:42.046536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:42.046536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:42.046536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:42.046536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 243/520] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:42.046536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5b70f-ca90-45c7-9f5f-e536a4436a4c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.054377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:42.064856 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e71f7-13a0-4b44-a95a-8033318a1dbc None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.101179 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52976b-2f27-4ec6-b10b-9fe63786100e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.101779 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52976b-2f27-4ec6-b10b-9fe63786100e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.102073 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52976b-2f27-4ec6-b10b-9fe63786100e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.102342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52976b-2f27-4ec6-b10b-9fe63786100e None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.103020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52976b-2f27-4ec6-b10b-9fe63786100e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:42.103292 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52976b-2f27-4ec6-b10b-9fe63786100e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.104721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52976b-2f27-4ec6-b10b-9fe63786100e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.116454 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 278/521] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:42.117246 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52976b-2f27-4ec6-b10b-9fe63786100e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.164533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5b70f-ca90-45c7-9f5f-e536a4436a4c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.199607 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad9b4a-a5f9-4fa8-8a04-00749e6a2294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.200382 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad9b4a-a5f9-4fa8-8a04-00749e6a2294 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.200660 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad9b4a-a5f9-4fa8-8a04-00749e6a2294 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.200924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad9b4a-a5f9-4fa8-8a04-00749e6a2294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.209008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:42.214992 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad9b4a-a5f9-4fa8-8a04-00749e6a2294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.216567 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad9b4a-a5f9-4fa8-8a04-00749e6a2294 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.220400 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52976b-2f27-4ec6-b10b-9fe63786100e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-b4ad9b4a-a5f9-4fa8-8a04-00749e6a2294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:42.230875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:42.231810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:42.232670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:42.232670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:42.232670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:42.232670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 244/522] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:42.232892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad9b4a-a5f9-4fa8-8a04-00749e6a2294 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.261070 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ddb5-a09e-49e3-b7d2-104c7ebf69ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.261708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ddb5-a09e-49e3-b7d2-104c7ebf69ac None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.261985 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ddb5-a09e-49e3-b7d2-104c7ebf69ac None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.262258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ddb5-a09e-49e3-b7d2-104c7ebf69ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.268985 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307ddb5-a09e-49e3-b7d2-104c7ebf69ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.270574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307ddb5-a09e-49e3-b7d2-104c7ebf69ac None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-9307ddb5-a09e-49e3-b7d2-104c7ebf69ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:42.282620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:42.283572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:42.284662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:42.284662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:42.284662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:42.284662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:42.284662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:42.284662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:42.284662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:42.284662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:42.288743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 279/523] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:42.289587 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ddb5-a09e-49e3-b7d2-104c7ebf69ac None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.305134 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:42.339556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad9b4a-a5f9-4fa8-8a04-00749e6a2294 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.373227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca3e8c-7898-498b-bb1e-3d11bcc8023f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.373835 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca3e8c-7898-498b-bb1e-3d11bcc8023f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.374115 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca3e8c-7898-498b-bb1e-3d11bcc8023f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.374386 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca3e8c-7898-498b-bb1e-3d11bcc8023f None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.375036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca3e8c-7898-498b-bb1e-3d11bcc8023f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:42.375371 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca3e8c-7898-498b-bb1e-3d11bcc8023f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.376873 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca3e8c-7898-498b-bb1e-3d11bcc8023f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.389200 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 245/524] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:42.389991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca3e8c-7898-498b-bb1e-3d11bcc8023f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.405258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ddb5-a09e-49e3-b7d2-104c7ebf69ac None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.446013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98ab77-5c22-4ff9-94c3-0c928cc689e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.446656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98ab77-5c22-4ff9-94c3-0c928cc689e3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.446936 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98ab77-5c22-4ff9-94c3-0c928cc689e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.447231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98ab77-5c22-4ff9-94c3-0c928cc689e3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.447354 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:42.456111 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98ab77-5c22-4ff9-94c3-0c928cc689e3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.457579 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98ab77-5c22-4ff9-94c3-0c928cc689e3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-bc98ab77-5c22-4ff9-94c3-0c928cc689e3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:42.469963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:42.470871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:42.471792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 280/525] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:42.472428 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98ab77-5c22-4ff9-94c3-0c928cc689e3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.508026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca3e8c-7898-498b-bb1e-3d11bcc8023f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.544298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291f34f-fdb5-4592-9a47-0ed9df4308aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.545048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291f34f-fdb5-4592-9a47-0ed9df4308aa None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.545323 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291f34f-fdb5-4592-9a47-0ed9df4308aa None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.545589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291f34f-fdb5-4592-9a47-0ed9df4308aa None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.546230 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291f34f-fdb5-4592-9a47-0ed9df4308aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:42.546526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291f34f-fdb5-4592-9a47-0ed9df4308aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.547931 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291f34f-fdb5-4592-9a47-0ed9df4308aa None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.554398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:42.559327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 246/526] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:42.560350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291f34f-fdb5-4592-9a47-0ed9df4308aa None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.591064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98ab77-5c22-4ff9-94c3-0c928cc689e3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.630169 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6e686-2b46-4501-8ef8-eb0238f21d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.630834 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6e686-2b46-4501-8ef8-eb0238f21d9c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.631098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6e686-2b46-4501-8ef8-eb0238f21d9c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.631323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6e686-2b46-4501-8ef8-eb0238f21d9c None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.632002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6e686-2b46-4501-8ef8-eb0238f21d9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:42.632306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6e686-2b46-4501-8ef8-eb0238f21d9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.633736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6e686-2b46-4501-8ef8-eb0238f21d9c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.651007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 281/527] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:42.651933 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6e686-2b46-4501-8ef8-eb0238f21d9c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.674101 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291f34f-fdb5-4592-9a47-0ed9df4308aa None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.684967 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:42.711433 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0219-67cb-4659-8db8-7f17c2647da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.712163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0219-67cb-4659-8db8-7f17c2647da7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.712672 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0219-67cb-4659-8db8-7f17c2647da7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.712776 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0219-67cb-4659-8db8-7f17c2647da7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.719372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0219-67cb-4659-8db8-7f17c2647da7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.720827 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0219-67cb-4659-8db8-7f17c2647da7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-73dd0219-67cb-4659-8db8-7f17c2647da7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:42.733962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:42.734937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 247/528] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:42.735861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0219-67cb-4659-8db8-7f17c2647da7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.755677 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6e686-2b46-4501-8ef8-eb0238f21d9c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.786443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:42.791855 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9bcf6-13e1-4f6e-9851-0d4ddfd8ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.792460 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9bcf6-13e1-4f6e-9851-0d4ddfd8ea2e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.792733 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9bcf6-13e1-4f6e-9851-0d4ddfd8ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.792994 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9bcf6-13e1-4f6e-9851-0d4ddfd8ea2e None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.793709 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9bcf6-13e1-4f6e-9851-0d4ddfd8ea2e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:42.794004 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9bcf6-13e1-4f6e-9851-0d4ddfd8ea2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.795507 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9bcf6-13e1-4f6e-9851-0d4ddfd8ea2e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.817519 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 282/529] 10.222.0.16 () {56 vars in 1139 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:42.817519 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9bcf6-13e1-4f6e-9851-0d4ddfd8ea2e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.849192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0219-67cb-4659-8db8-7f17c2647da7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.886824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6032fb0-d816-4ff4-9622-75f4a9ffdede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.887476 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6032fb0-d816-4ff4-9622-75f4a9ffdede None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.887750 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6032fb0-d816-4ff4-9622-75f4a9ffdede None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.888012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6032fb0-d816-4ff4-9622-75f4a9ffdede None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.896696 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6032fb0-d816-4ff4-9622-75f4a9ffdede None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.898847 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6032fb0-d816-4ff4-9622-75f4a9ffdede None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-a6032fb0-d816-4ff4-9622-75f4a9ffdede None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:42.914436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:42.915331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:42.921232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:42.921232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:42.921232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:42.921232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:42.921232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:42.921232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:42.921232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:42.921232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:42.921670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9bcf6-13e1-4f6e-9851-0d4ddfd8ea2e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:42.921746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 248/530] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:42.921746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6032fb0-d816-4ff4-9622-75f4a9ffdede None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:42.955474 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4980b64-6f61-4890-b8c1-4a172113317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:42.956130 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4980b64-6f61-4890-b8c1-4a172113317b None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:42.956376 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4980b64-6f61-4890-b8c1-4a172113317b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:42.956646 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4980b64-6f61-4890-b8c1-4a172113317b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:42.957399 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4980b64-6f61-4890-b8c1-4a172113317b None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=0bb80229c3234d5f9599b16870c69934) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:42.957675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4980b64-6f61-4890-b8c1-4a172113317b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:42.959202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4980b64-6f61-4890-b8c1-4a172113317b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:42.963882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:42.964781 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 283/531] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/role_assignments?user.id=0bb80229c3234d5f9599b16870c69934&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:42.965698 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4980b64-6f61-4890-b8c1-4a172113317b None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:43.043520 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:43.093717 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6032fb0-d816-4ff4-9622-75f4a9ffdede None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:43.099825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4980b64-6f61-4890-b8c1-4a172113317b None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:43.134254 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bfb43-cac3-4cb1-b162-badb0e514ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:43.134860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bfb43-cac3-4cb1-b162-badb0e514ff4 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:43.135237 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bfb43-cac3-4cb1-b162-badb0e514ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:43.135515 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bfb43-cac3-4cb1-b162-badb0e514ff4 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:43.136323 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bfb43-cac3-4cb1-b162-badb0e514ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:43.136540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bfb43-cac3-4cb1-b162-badb0e514ff4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:43.138033 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bfb43-cac3-4cb1-b162-badb0e514ff4 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:43.145664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf8c3a-65ef-40ce-8647-ffdefde18c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:43.146850 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf8c3a-65ef-40ce-8647-ffdefde18c03 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:43.147368 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf8c3a-65ef-40ce-8647-ffdefde18c03 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:43.147931 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf8c3a-65ef-40ce-8647-ffdefde18c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:43.164192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 249/532] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:43.165127 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bfb43-cac3-4cb1-b162-badb0e514ff4 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:43.283023 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:43.285230 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf8c3a-65ef-40ce-8647-ffdefde18c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:43.287045 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf8c3a-65ef-40ce-8647-ffdefde18c03 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:43.293062 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bfb43-cac3-4cb1-b162-badb0e514ff4 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:43.331287 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722099c-5c52-4feb-848d-c8af8c84bb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:43.332149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722099c-5c52-4feb-848d-c8af8c84bb44 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:43.332430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722099c-5c52-4feb-848d-c8af8c84bb44 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:43.332703 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722099c-5c52-4feb-848d-c8af8c84bb44 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:43.333452 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722099c-5c52-4feb-848d-c8af8c84bb44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:43.334012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722099c-5c52-4feb-848d-c8af8c84bb44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:43.335683 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722099c-5c52-4feb-848d-c8af8c84bb44 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:43.349159 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 250/533] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:43.350011 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722099c-5c52-4feb-848d-c8af8c84bb44 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:43.414706 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:43.479078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722099c-5c52-4feb-848d-c8af8c84bb44 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:43.495873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 284/534] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:43.495873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf8c3a-65ef-40ce-8647-ffdefde18c03 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:43.526766 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089307b-bc27-406f-968e-9fec8b010322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:43.527487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089307b-bc27-406f-968e-9fec8b010322 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:43.527953 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089307b-bc27-406f-968e-9fec8b010322 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:43.528214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089307b-bc27-406f-968e-9fec8b010322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:43.528942 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089307b-bc27-406f-968e-9fec8b010322 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=efbe019e4d4646428bed474cef4009b8) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:43.529243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089307b-bc27-406f-968e-9fec8b010322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:43.530838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089307b-bc27-406f-968e-9fec8b010322 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:43.544146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 251/535] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:43 2022] GET /identity/v3/role_assignments?user.id=efbe019e4d4646428bed474cef4009b8&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:43.544146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5001af-b6cd-43be-8182-8ba64b2046a9 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:43.544146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5001af-b6cd-43be-8182-8ba64b2046a9 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:43.544146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5001af-b6cd-43be-8182-8ba64b2046a9 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:43.544146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 252/536] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:43.544146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5001af-b6cd-43be-8182-8ba64b2046a9 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:43.562329 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:43.662064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf8c3a-65ef-40ce-8647-ffdefde18c03 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:43.671217 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5001af-b6cd-43be-8182-8ba64b2046a9 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:43.709981 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259377d0-b62d-4502-9bca-1db0cfdc9ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:43.710591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259377d0-b62d-4502-9bca-1db0cfdc9ea7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:43.710861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259377d0-b62d-4502-9bca-1db0cfdc9ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:43.711191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259377d0-b62d-4502-9bca-1db0cfdc9ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:43.711867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259377d0-b62d-4502-9bca-1db0cfdc9ea7 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=d3250e8f23c64f26b4e862090f7f2e18) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:43.712163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259377d0-b62d-4502-9bca-1db0cfdc9ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:43.712480 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbee07-7fc4-47ff-9723-4689b2b9b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:43.713111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbee07-7fc4-47ff-9723-4689b2b9b09b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:43.713382 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbee07-7fc4-47ff-9723-4689b2b9b09b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:43.713597 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259377d0-b62d-4502-9bca-1db0cfdc9ea7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:43.713689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbee07-7fc4-47ff-9723-4689b2b9b09b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:43.720154 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 285/537] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:43 2022] GET /identity/v3/role_assignments?user.id=d3250e8f23c64f26b4e862090f7f2e18&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:43.721195 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f706d9-2cfa-4fed-a6f8-7b2f5b9141ff None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:43.722498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f706d9-2cfa-4fed-a6f8-7b2f5b9141ff None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:43.722498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f706d9-2cfa-4fed-a6f8-7b2f5b9141ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:43.725697 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbee07-7fc4-47ff-9723-4689b2b9b09b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:43.727232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbee07-7fc4-47ff-9723-4689b2b9b09b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-4abbee07-7fc4-47ff-9723-4689b2b9b09b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:43.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:43.743866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:43.744913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:43.744913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:43.744913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:43.744913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 253/538] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:43.747206 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-93f706d9-2cfa-4fed-a6f8-7b2f5b9141ff None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:43.748661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-93f706d9-2cfa-4fed-a6f8-7b2f5b9141ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:43.749883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbee07-7fc4-47ff-9723-4689b2b9b09b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:43.767532 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:43.873872 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:43.892517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 286/539] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:43.914384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbee07-7fc4-47ff-9723-4689b2b9b09b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:43.951449 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1caea-2ddb-4154-87d8-4392b3607f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:43.952125 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1caea-2ddb-4154-87d8-4392b3607f87 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:43.952402 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1caea-2ddb-4154-87d8-4392b3607f87 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:43.952672 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1caea-2ddb-4154-87d8-4392b3607f87 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:43.953389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1caea-2ddb-4154-87d8-4392b3607f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:43.953693 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1caea-2ddb-4154-87d8-4392b3607f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:43.955517 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1caea-2ddb-4154-87d8-4392b3607f87 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:43.968115 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 254/540] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:43.973316 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f706d9-2cfa-4fed-a6f8-7b2f5b9141ff None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:44.105834 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f706d9-2cfa-4fed-a6f8-7b2f5b9141ff None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:44.156753 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb66a77-6389-426a-9741-90c41ed5bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:44.157398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb66a77-6389-426a-9741-90c41ed5bf3c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:44.157665 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb66a77-6389-426a-9741-90c41ed5bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:44.157925 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb66a77-6389-426a-9741-90c41ed5bf3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:44.165199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb66a77-6389-426a-9741-90c41ed5bf3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:44.166597 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb66a77-6389-426a-9741-90c41ed5bf3c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-5eb66a77-6389-426a-9741-90c41ed5bf3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:44.177963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:44.178829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:44.179899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:44.179899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:44.179899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:44.179899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:44.179899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:44.179899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:44.179899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:44.179899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:44.179899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 287/541] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:44.188110 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb66a77-6389-426a-9741-90c41ed5bf3c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:44.188238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1caea-2ddb-4154-87d8-4392b3607f87 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:44.199450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:44.290759 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb66a77-6389-426a-9741-90c41ed5bf3c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:44.329288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99370a8d-f16c-4b1e-a3ee-185878a0468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:44.329919 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99370a8d-f16c-4b1e-a3ee-185878a0468e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:44.330188 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99370a8d-f16c-4b1e-a3ee-185878a0468e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:44.330451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99370a8d-f16c-4b1e-a3ee-185878a0468e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:44.340924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99370a8d-f16c-4b1e-a3ee-185878a0468e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-99370a8d-f16c-4b1e-a3ee-185878a0468e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:44.345821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:44.347466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:44.348348 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 288/542] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:44.352911 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99370a8d-f16c-4b1e-a3ee-185878a0468e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:44.383634 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1caea-2ddb-4154-87d8-4392b3607f87 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:44.420267 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:44.434706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:44.457108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aada84-75f2-4c4c-a80e-4602b89add7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:44.457923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aada84-75f2-4c4c-a80e-4602b89add7a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:44.458198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aada84-75f2-4c4c-a80e-4602b89add7a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:44.458762 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aada84-75f2-4c4c-a80e-4602b89add7a None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:44.459786 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aada84-75f2-4c4c-a80e-4602b89add7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:44.460088 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aada84-75f2-4c4c-a80e-4602b89add7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:44.461180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99370a8d-f16c-4b1e-a3ee-185878a0468e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:44.461516 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aada84-75f2-4c4c-a80e-4602b89add7a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:44.477283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 255/543] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:44.484130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aada84-75f2-4c4c-a80e-4602b89add7a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:44.543787 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d6e95-e0bc-49e5-908b-b144bf3fa57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:44.544407 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d6e95-e0bc-49e5-908b-b144bf3fa57c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:44.544680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d6e95-e0bc-49e5-908b-b144bf3fa57c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:44.544942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d6e95-e0bc-49e5-908b-b144bf3fa57c None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:44.545610 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d6e95-e0bc-49e5-908b-b144bf3fa57c None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:44.545905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d6e95-e0bc-49e5-908b-b144bf3fa57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:44.547311 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d6e95-e0bc-49e5-908b-b144bf3fa57c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:44.566264 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 289/544] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:44.571449 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d6e95-e0bc-49e5-908b-b144bf3fa57c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:44.630405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aada84-75f2-4c4c-a80e-4602b89add7a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:44.665809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f6da3-0748-493b-b319-b593df066adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:44.666410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f6da3-0748-493b-b319-b593df066adb None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:44.666679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f6da3-0748-493b-b319-b593df066adb None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:44.666943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f6da3-0748-493b-b319-b593df066adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:44.667701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f6da3-0748-493b-b319-b593df066adb None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=56bc4fdd14d147ef86b96786da2855f5) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:44.668004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f6da3-0748-493b-b319-b593df066adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:44.669428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f6da3-0748-493b-b319-b593df066adb None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:44.675435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 256/545] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:44 2022] GET /identity/v3/role_assignments?user.id=56bc4fdd14d147ef86b96786da2855f5&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:44.704180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d6e95-e0bc-49e5-908b-b144bf3fa57c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:44.737064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:44.744956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9b28c-3ce1-4837-9cc2-41959c9e4d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:44.745598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9b28c-3ce1-4837-9cc2-41959c9e4d96 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:44.745870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9b28c-3ce1-4837-9cc2-41959c9e4d96 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:44.746129 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9b28c-3ce1-4837-9cc2-41959c9e4d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:44.759395 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9b28c-3ce1-4837-9cc2-41959c9e4d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:44.760872 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9b28c-3ce1-4837-9cc2-41959c9e4d96 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-17b9b28c-3ce1-4837-9cc2-41959c9e4d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:44.776530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:44.777485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:44.779299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:44.779299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:44.779299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:44.779299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 290/546] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:44.783752 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f6da3-0748-493b-b319-b593df066adb None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:44.859580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:44.911832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f6da3-0748-493b-b319-b593df066adb None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:44.945337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc6795-32f3-4ff0-b852-b45123794d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:44.945861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc6795-32f3-4ff0-b852-b45123794d73 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:44.946136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc6795-32f3-4ff0-b852-b45123794d73 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:44.946399 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc6795-32f3-4ff0-b852-b45123794d73 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:44.947041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc6795-32f3-4ff0-b852-b45123794d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:44.947365 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc6795-32f3-4ff0-b852-b45123794d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:44.948751 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc6795-32f3-4ff0-b852-b45123794d73 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:44.961503 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 257/547] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:44.966675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9b28c-3ce1-4837-9cc2-41959c9e4d96 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:45.112615 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9b28c-3ce1-4837-9cc2-41959c9e4d96 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:45.164833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10ac1b-df52-4e51-978e-3e5f133676bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:45.165480 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10ac1b-df52-4e51-978e-3e5f133676bd None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:45.165755 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10ac1b-df52-4e51-978e-3e5f133676bd None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:45.166025 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10ac1b-df52-4e51-978e-3e5f133676bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:45.177251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8a3e1-41ba-45aa-b676-0371fff2b019 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:45.177530 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8a3e1-41ba-45aa-b676-0371fff2b019 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:45.177782 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8a3e1-41ba-45aa-b676-0371fff2b019 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:45.180208 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10ac1b-df52-4e51-978e-3e5f133676bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:45.181727 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10ac1b-df52-4e51-978e-3e5f133676bd None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:45.183173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 258/548] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:45.184328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3192c-6d27-434b-a803-2286158cd5ba None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:45.184603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3192c-6d27-434b-a803-2286158cd5ba None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:45.184871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3192c-6d27-434b-a803-2286158cd5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-cb10ac1b-df52-4e51-978e-3e5f133676bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:45.196535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:45.199334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:36:45.201644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:45.208119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 291/549] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:36:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:45.208119 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10ac1b-df52-4e51-978e-3e5f133676bd None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:45.217702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-f0a3192c-6d27-434b-a803-2286158cd5ba None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:45.218876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-f0a3192c-6d27-434b-a803-2286158cd5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:45.286702 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:45.345014 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 259/550] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:45.351016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10ac1b-df52-4e51-978e-3e5f133676bd None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:45.384435 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43eec7-a5fd-4898-b3bb-a5700164bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:45.385047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43eec7-a5fd-4898-b3bb-a5700164bc0c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:45.385316 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43eec7-a5fd-4898-b3bb-a5700164bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:45.385580 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43eec7-a5fd-4898-b3bb-a5700164bc0c None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:45.386296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43eec7-a5fd-4898-b3bb-a5700164bc0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:45.386591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43eec7-a5fd-4898-b3bb-a5700164bc0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:45.388047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43eec7-a5fd-4898-b3bb-a5700164bc0c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:45.405727 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 292/551] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Aug 30 11:36:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:45.411655 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3192c-6d27-434b-a803-2286158cd5ba None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:45.469790 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:45.536231 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3192c-6d27-434b-a803-2286158cd5ba None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:45.584661 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77cf80-def3-4dc8-9020-2e6115c9cb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:45.585540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77cf80-def3-4dc8-9020-2e6115c9cb3b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:45.585967 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77cf80-def3-4dc8-9020-2e6115c9cb3b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:45.586369 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77cf80-def3-4dc8-9020-2e6115c9cb3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:45.654693 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77cf80-def3-4dc8-9020-2e6115c9cb3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:45.656847 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77cf80-def3-4dc8-9020-2e6115c9cb3b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:45.702684 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:45.716560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43eec7-a5fd-4898-b3bb-a5700164bc0c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:45.806447 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 260/552] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 11:36:45.820851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77cf80-def3-4dc8-9020-2e6115c9cb3b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:45.828141 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:45.832393 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43eec7-a5fd-4898-b3bb-a5700164bc0c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:45.876810 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93d9e4-3b0f-44bf-bbee-818b4b28ed26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:45.877452 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93d9e4-3b0f-44bf-bbee-818b4b28ed26 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:45.877724 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93d9e4-3b0f-44bf-bbee-818b4b28ed26 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:45.877984 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93d9e4-3b0f-44bf-bbee-818b4b28ed26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:45.890890 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93d9e4-3b0f-44bf-bbee-818b4b28ed26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-fc93d9e4-3b0f-44bf-bbee-818b4b28ed26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:45.896921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:45.898002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:45.898863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 293/553] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:45.908026 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93d9e4-3b0f-44bf-bbee-818b4b28ed26 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:45.940808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77cf80-def3-4dc8-9020-2e6115c9cb3b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:45.980470 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec4f12-b1cf-4e3a-9ce3-bc1664958e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:45.981613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4f12-b1cf-4e3a-9ce3-bc1664958e2b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:45.982095 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4f12-b1cf-4e3a-9ce3-bc1664958e2b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:45.982565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4f12-b1cf-4e3a-9ce3-bc1664958e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:45.992101 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:45.995637 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec4f12-b1cf-4e3a-9ce3-bc1664958e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:45.997877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec4f12-b1cf-4e3a-9ce3-bc1664958e2b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-7dec4f12-b1cf-4e3a-9ce3-bc1664958e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:46.015251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:46.016220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:46.017189 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:46.017189 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:46.017189 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:46.019359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 261/554] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:46.020821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301c0a2-0ab5-472f-97f7-84b48f398734 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.021316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301c0a2-0ab5-472f-97f7-84b48f398734 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.021778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301c0a2-0ab5-472f-97f7-84b48f398734 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.023703 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 262/555] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:46.024769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301c0a2-0ab5-472f-97f7-84b48f398734 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.037412 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93d9e4-3b0f-44bf-bbee-818b4b28ed26 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.073482 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316c205-4a90-4aa9-b189-5625b347ecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.073911 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316c205-4a90-4aa9-b189-5625b347ecfb None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.074180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316c205-4a90-4aa9-b189-5625b347ecfb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.074444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316c205-4a90-4aa9-b189-5625b347ecfb None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.075099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316c205-4a90-4aa9-b189-5625b347ecfb None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:46.075418 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316c205-4a90-4aa9-b189-5625b347ecfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:46.076982 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316c205-4a90-4aa9-b189-5625b347ecfb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.085721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 294/556] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:46.086972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d552559-05b4-4aeb-a011-e1d098b011b4 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.087290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d552559-05b4-4aeb-a011-e1d098b011b4 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.087559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d552559-05b4-4aeb-a011-e1d098b011b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.106486 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-6d552559-05b4-4aeb-a011-e1d098b011b4 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:46.107705 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-6d552559-05b4-4aeb-a011-e1d098b011b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:46.124740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:46.148797 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301c0a2-0ab5-472f-97f7-84b48f398734 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.175565 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 295/557] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:46.176505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d552559-05b4-4aeb-a011-e1d098b011b4 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.186151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049ca99-8df9-4b2b-b6f0-9d30abbdfaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.187141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049ca99-8df9-4b2b-b6f0-9d30abbdfaa3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.187639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049ca99-8df9-4b2b-b6f0-9d30abbdfaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.188222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049ca99-8df9-4b2b-b6f0-9d30abbdfaa3 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.189065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049ca99-8df9-4b2b-b6f0-9d30abbdfaa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:46.189564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049ca99-8df9-4b2b-b6f0-9d30abbdfaa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:46.191258 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049ca99-8df9-4b2b-b6f0-9d30abbdfaa3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.205154 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 263/558] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:46.216165 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049ca99-8df9-4b2b-b6f0-9d30abbdfaa3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.277730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d552559-05b4-4aeb-a011-e1d098b011b4 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.300828 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:46.345713 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5442d56-2fe2-40c6-8765-026094b5dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.346343 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5442d56-2fe2-40c6-8765-026094b5dedc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.346539 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5442d56-2fe2-40c6-8765-026094b5dedc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.346787 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5442d56-2fe2-40c6-8765-026094b5dedc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.353024 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049ca99-8df9-4b2b-b6f0-9d30abbdfaa3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.357948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5442d56-2fe2-40c6-8765-026094b5dedc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:46.359937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5442d56-2fe2-40c6-8765-026094b5dedc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-d5442d56-2fe2-40c6-8765-026094b5dedc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:46.374721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:46.375948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:46.376875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:46.376875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:46.376875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:46.376875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 296/559] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:46.377449 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f19c2-4782-430d-83ee-ca17af0c2839 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.377718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f19c2-4782-430d-83ee-ca17af0c2839 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.377995 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f19c2-4782-430d-83ee-ca17af0c2839 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.383450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 297/560] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:46.384613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15bb42-b6e6-46b7-b3c3-6989e5678e1e None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.384887 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15bb42-b6e6-46b7-b3c3-6989e5678e1e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.385149 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15bb42-b6e6-46b7-b3c3-6989e5678e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.396073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5275ec9-7447-4288-ad0b-13f59e442f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.397070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5275ec9-7447-4288-ad0b-13f59e442f1a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.397558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5275ec9-7447-4288-ad0b-13f59e442f1a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.398071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5275ec9-7447-4288-ad0b-13f59e442f1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.404461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-9c15bb42-b6e6-46b7-b3c3-6989e5678e1e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:46.405628 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-9c15bb42-b6e6-46b7-b3c3-6989e5678e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:46.406691 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5275ec9-7447-4288-ad0b-13f59e442f1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:46.408515 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5275ec9-7447-4288-ad0b-13f59e442f1a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.409146 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-c5275ec9-7447-4288-ad0b-13f59e442f1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:46.422113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:46.423396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:46.424348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:46.424348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:46.424348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:46.424348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:46.424348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:46.424348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:46.424348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:46.424348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:46.426619 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 264/561] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:46.427665 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5275ec9-7447-4288-ad0b-13f59e442f1a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.482781 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 298/562] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:46.483800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15bb42-b6e6-46b7-b3c3-6989e5678e1e None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.534991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5275ec9-7447-4288-ad0b-13f59e442f1a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.548856 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:46.573169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d04003-a7c9-4dbc-b25c-47959068d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.574142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d04003-a7c9-4dbc-b25c-47959068d726 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.574643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d04003-a7c9-4dbc-b25c-47959068d726 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.575140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d04003-a7c9-4dbc-b25c-47959068d726 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.576012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d04003-a7c9-4dbc-b25c-47959068d726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:46.576514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d04003-a7c9-4dbc-b25c-47959068d726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:46.578146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d04003-a7c9-4dbc-b25c-47959068d726 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.588417 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15bb42-b6e6-46b7-b3c3-6989e5678e1e None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.592227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 265/563] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:46.593331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d04003-a7c9-4dbc-b25c-47959068d726 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.629107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7564471-50ea-4553-bd0e-9532b2689e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.629685 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7564471-50ea-4553-bd0e-9532b2689e40 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.629959 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7564471-50ea-4553-bd0e-9532b2689e40 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.630226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7564471-50ea-4553-bd0e-9532b2689e40 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.631427 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7564471-50ea-4553-bd0e-9532b2689e40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:46.631427 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7564471-50ea-4553-bd0e-9532b2689e40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:46.632670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7564471-50ea-4553-bd0e-9532b2689e40 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.642702 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:46.650144 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 299/564] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:46.651156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7564471-50ea-4553-bd0e-9532b2689e40 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.705203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d04003-a7c9-4dbc-b25c-47959068d726 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.741463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e563b1-6cb8-481b-b970-528c4f9c3724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.742164 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e563b1-6cb8-481b-b970-528c4f9c3724 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.742427 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e563b1-6cb8-481b-b970-528c4f9c3724 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.742694 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e563b1-6cb8-481b-b970-528c4f9c3724 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.756038 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e563b1-6cb8-481b-b970-528c4f9c3724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-30e563b1-6cb8-481b-b970-528c4f9c3724 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:46.761411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:46.762319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:46.763145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 266/565] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:46.763844 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e563b1-6cb8-481b-b970-528c4f9c3724 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.766221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7564471-50ea-4553-bd0e-9532b2689e40 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.797920 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:46.805186 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e35fd-613b-4d8c-b906-b7a5f3cb6fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.805826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e35fd-613b-4d8c-b906-b7a5f3cb6fe3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.806098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e35fd-613b-4d8c-b906-b7a5f3cb6fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.806365 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e35fd-613b-4d8c-b906-b7a5f3cb6fe3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.814203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e35fd-613b-4d8c-b906-b7a5f3cb6fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:46.815735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e35fd-613b-4d8c-b906-b7a5f3cb6fe3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-126e35fd-613b-4d8c-b906-b7a5f3cb6fe3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:46.826930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:46.828007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:36:46.829725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:46.833134 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 300/566] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:46.833694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e35fd-613b-4d8c-b906-b7a5f3cb6fe3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.873812 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e563b1-6cb8-481b-b970-528c4f9c3724 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.889964 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:46.906729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf75a34-b663-4549-bb3c-d70a34e87da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.907366 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf75a34-b663-4549-bb3c-d70a34e87da0 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.907642 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf75a34-b663-4549-bb3c-d70a34e87da0 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.907907 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf75a34-b663-4549-bb3c-d70a34e87da0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.908629 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf75a34-b663-4549-bb3c-d70a34e87da0 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=8bbb8fb036684305bc00290c519939ec) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:46.908932 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf75a34-b663-4549-bb3c-d70a34e87da0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:46.910377 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf75a34-b663-4549-bb3c-d70a34e87da0 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.915833 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 267/567] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/role_assignments?user.id=8bbb8fb036684305bc00290c519939ec&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:46.916602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf75a34-b663-4549-bb3c-d70a34e87da0 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:46.936545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e35fd-613b-4d8c-b906-b7a5f3cb6fe3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:46.975290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0bbc3-da78-40fe-9963-740fefbe29d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:46.975913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0bbc3-da78-40fe-9963-740fefbe29d0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:46.976195 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0bbc3-da78-40fe-9963-740fefbe29d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:46.976461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0bbc3-da78-40fe-9963-740fefbe29d0 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:46.977136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0bbc3-da78-40fe-9963-740fefbe29d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:46.977437 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0bbc3-da78-40fe-9963-740fefbe29d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:46.979199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0bbc3-da78-40fe-9963-740fefbe29d0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:46.992553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 301/568] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:46.992553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0bbc3-da78-40fe-9963-740fefbe29d0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.034669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:47.041254 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf75a34-b663-4549-bb3c-d70a34e87da0 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.078164 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c088c8-ec14-4ea7-8619-8928bf8459f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.079185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c088c8-ec14-4ea7-8619-8928bf8459f2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.079463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c088c8-ec14-4ea7-8619-8928bf8459f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.079732 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c088c8-ec14-4ea7-8619-8928bf8459f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.090332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c088c8-ec14-4ea7-8619-8928bf8459f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-50c088c8-ec14-4ea7-8619-8928bf8459f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:47.095589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:47.096507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:47.097344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 268/569] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:47.098026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6de55-6943-4765-9e9d-97182a578c93 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.098092 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6de55-6943-4765-9e9d-97182a578c93 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.098732 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6de55-6943-4765-9e9d-97182a578c93 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.103600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 269/570] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:47.104362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6de55-6943-4765-9e9d-97182a578c93 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.121774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0bbc3-da78-40fe-9963-740fefbe29d0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.148288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:47.167704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f19186-23b8-46bf-b3a7-3734d4a72698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.168316 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f19186-23b8-46bf-b3a7-3734d4a72698 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.168591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f19186-23b8-46bf-b3a7-3734d4a72698 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.168857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f19186-23b8-46bf-b3a7-3734d4a72698 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.169619 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f19186-23b8-46bf-b3a7-3734d4a72698 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:47.169869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f19186-23b8-46bf-b3a7-3734d4a72698 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:47.171414 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f19186-23b8-46bf-b3a7-3734d4a72698 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:47.193101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 302/571] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Aug 30 11:36:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:47.193101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f19186-23b8-46bf-b3a7-3734d4a72698 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.214298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6de55-6943-4765-9e9d-97182a578c93 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.250183 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0df8a6-90e2-49d1-ae48-c6964f9a4383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.251103 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0df8a6-90e2-49d1-ae48-c6964f9a4383 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.251539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0df8a6-90e2-49d1-ae48-c6964f9a4383 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.251949 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0df8a6-90e2-49d1-ae48-c6964f9a4383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.263130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0df8a6-90e2-49d1-ae48-c6964f9a4383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:47.264867 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0df8a6-90e2-49d1-ae48-c6964f9a4383 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-fd0df8a6-90e2-49d1-ae48-c6964f9a4383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:47.280185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:47.281321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:47.282478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:47.282478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:47.282478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:47.288721 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 270/572] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:47.289976 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fef058-79c4-4765-9c70-d6562f9e3ae0 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.290255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fef058-79c4-4765-9c70-d6562f9e3ae0 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.290524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fef058-79c4-4765-9c70-d6562f9e3ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.294244 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f19186-23b8-46bf-b3a7-3734d4a72698 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.310196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-04fef058-79c4-4765-9c70-d6562f9e3ae0 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:47.311522 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-04fef058-79c4-4765-9c70-d6562f9e3ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:47.313612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:47.333988 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ded5c-edd4-4c43-85f5-263f23ca002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.334590 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ded5c-edd4-4c43-85f5-263f23ca002c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.334863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ded5c-edd4-4c43-85f5-263f23ca002c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.335155 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ded5c-edd4-4c43-85f5-263f23ca002c None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.335833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ded5c-edd4-4c43-85f5-263f23ca002c None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:47.336124 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ded5c-edd4-4c43-85f5-263f23ca002c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:47.337499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ded5c-edd4-4c43-85f5-263f23ca002c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:47.347170 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 303/573] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:47.347477 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ded5c-edd4-4c43-85f5-263f23ca002c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.386559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:47.403219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 271/574] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:47.403219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fef058-79c4-4765-9c70-d6562f9e3ae0 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.473853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ded5c-edd4-4c43-85f5-263f23ca002c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.512278 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f594-adc7-4347-8137-71586415c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.512923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f594-adc7-4347-8137-71586415c2fd None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.513197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f594-adc7-4347-8137-71586415c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.513462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f594-adc7-4347-8137-71586415c2fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.522381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fef058-79c4-4765-9c70-d6562f9e3ae0 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.526106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8f594-adc7-4347-8137-71586415c2fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:47.527779 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8f594-adc7-4347-8137-71586415c2fd None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-6ff8f594-adc7-4347-8137-71586415c2fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:47.544554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:47.545530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:47.546420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:47.546420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:47.546420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:47.546420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 304/575] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:47.547158 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f594-adc7-4347-8137-71586415c2fd None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.567160 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92453d-160d-48b4-9bef-8962391b0df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.568020 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92453d-160d-48b4-9bef-8962391b0df3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.568715 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92453d-160d-48b4-9bef-8962391b0df3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.569040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92453d-160d-48b4-9bef-8962391b0df3 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.569688 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92453d-160d-48b4-9bef-8962391b0df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:47.569993 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92453d-160d-48b4-9bef-8962391b0df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:47.571973 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92453d-160d-48b4-9bef-8962391b0df3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:47.580913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:47.588929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 272/576] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:47.589912 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92453d-160d-48b4-9bef-8962391b0df3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.646492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:47.660982 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f594-adc7-4347-8137-71586415c2fd None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.698200 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea92cfd-e7b0-4ffe-8552-91f32ffc0844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.698895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea92cfd-e7b0-4ffe-8552-91f32ffc0844 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.699425 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea92cfd-e7b0-4ffe-8552-91f32ffc0844 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.699703 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea92cfd-e7b0-4ffe-8552-91f32ffc0844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.702712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92453d-160d-48b4-9bef-8962391b0df3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.714386 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea92cfd-e7b0-4ffe-8552-91f32ffc0844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:47.715871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea92cfd-e7b0-4ffe-8552-91f32ffc0844 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-2ea92cfd-e7b0-4ffe-8552-91f32ffc0844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:47.729833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:47.730816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:47.732421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:47.732421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:47.732421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:47.732421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 305/577] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:47.732421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea92cfd-e7b0-4ffe-8552-91f32ffc0844 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.745953 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccf355-d205-4375-b513-cdf8a9cc5f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.746670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccf355-d205-4375-b513-cdf8a9cc5f53 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.746946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccf355-d205-4375-b513-cdf8a9cc5f53 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.747239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccf355-d205-4375-b513-cdf8a9cc5f53 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.747986 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccf355-d205-4375-b513-cdf8a9cc5f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:47.748284 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccf355-d205-4375-b513-cdf8a9cc5f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:47.749640 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccf355-d205-4375-b513-cdf8a9cc5f53 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:47.766929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 273/578] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:47.766929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccf355-d205-4375-b513-cdf8a9cc5f53 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.825543 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:47.841938 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea92cfd-e7b0-4ffe-8552-91f32ffc0844 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.869061 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccf355-d205-4375-b513-cdf8a9cc5f53 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:47.877507 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffe067-7043-44a6-95ce-8d28fc9684d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.878148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffe067-7043-44a6-95ce-8d28fc9684d0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.878426 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffe067-7043-44a6-95ce-8d28fc9684d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.878696 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffe067-7043-44a6-95ce-8d28fc9684d0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.887169 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffe067-7043-44a6-95ce-8d28fc9684d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:47.888653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffe067-7043-44a6-95ce-8d28fc9684d0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:47.891283 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-b9ffe067-7043-44a6-95ce-8d28fc9684d0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:47.901270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:47.902216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 306/579] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:47.903708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffe067-7043-44a6-95ce-8d28fc9684d0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:47.911837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd0d0d-cae0-4977-85a0-015739653ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:47.912488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd0d0d-cae0-4977-85a0-015739653ed3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:47.912772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd0d0d-cae0-4977-85a0-015739653ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:47.913152 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd0d0d-cae0-4977-85a0-015739653ed3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:47.925325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd0d0d-cae0-4977-85a0-015739653ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-fbdd0d0d-cae0-4977-85a0-015739653ed3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:47.930111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:47.931163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:47.932015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 274/580] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:47.932723 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd0d0d-cae0-4977-85a0-015739653ed3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.017159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffe067-7043-44a6-95ce-8d28fc9684d0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.036164 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd0d0d-cae0-4977-85a0-015739653ed3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.050438 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd39e71-c19d-4854-9787-5330ef841c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.051054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd39e71-c19d-4854-9787-5330ef841c6e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.051360 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd39e71-c19d-4854-9787-5330ef841c6e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.051691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd39e71-c19d-4854-9787-5330ef841c6e None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.052335 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd39e71-c19d-4854-9787-5330ef841c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:48.052636 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd39e71-c19d-4854-9787-5330ef841c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.053197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:48.054012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd39e71-c19d-4854-9787-5330ef841c6e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.066167 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 307/581] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:48.067060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd39e71-c19d-4854-9787-5330ef841c6e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.078606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e71cb-4611-499d-9655-e5bb67fa6181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.079589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e71cb-4611-499d-9655-e5bb67fa6181 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.079862 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e71cb-4611-499d-9655-e5bb67fa6181 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.080133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e71cb-4611-499d-9655-e5bb67fa6181 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.086756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e71cb-4611-499d-9655-e5bb67fa6181 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.088249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e71cb-4611-499d-9655-e5bb67fa6181 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-473e71cb-4611-499d-9655-e5bb67fa6181 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:48.100194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:48.101075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:48.101951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:48.101951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:48.101951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:48.101951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:48.101951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:48.101951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:48.101951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:48.101951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:48.101951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 275/582] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:48.102428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e71cb-4611-499d-9655-e5bb67fa6181 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.135848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:48.187527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd39e71-c19d-4854-9787-5330ef841c6e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.211898 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e71cb-4611-499d-9655-e5bb67fa6181 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.222875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc2006-e934-4b4d-a71b-56f68d97a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.223598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc2006-e934-4b4d-a71b-56f68d97a6dd None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.223872 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc2006-e934-4b4d-a71b-56f68d97a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.224158 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc2006-e934-4b4d-a71b-56f68d97a6dd None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.224884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc2006-e934-4b4d-a71b-56f68d97a6dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:48.225195 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc2006-e934-4b4d-a71b-56f68d97a6dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.226614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc2006-e934-4b4d-a71b-56f68d97a6dd None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.244876 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 308/583] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:48.247578 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d614145-2d4f-45dd-bc1b-3369b6bd063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.248186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d614145-2d4f-45dd-bc1b-3369b6bd063a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.248460 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d614145-2d4f-45dd-bc1b-3369b6bd063a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.248723 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d614145-2d4f-45dd-bc1b-3369b6bd063a None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.249472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d614145-2d4f-45dd-bc1b-3369b6bd063a None admin] RBAC: Adding query filter params (name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:48.249774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d614145-2d4f-45dd-bc1b-3369b6bd063a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.250054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc2006-e934-4b4d-a71b-56f68d97a6dd None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.251172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d614145-2d4f-45dd-bc1b-3369b6bd063a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.264294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 276/584] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:48.264294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d614145-2d4f-45dd-bc1b-3369b6bd063a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.305894 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:48.361043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc2006-e934-4b4d-a71b-56f68d97a6dd None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.373415 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d614145-2d4f-45dd-bc1b-3369b6bd063a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.380064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:48.397609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a6f3d-03f8-44e4-becb-250de76f00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.398343 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a6f3d-03f8-44e4-becb-250de76f00f6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.398679 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a6f3d-03f8-44e4-becb-250de76f00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.398952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a6f3d-03f8-44e4-becb-250de76f00f6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.406207 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a6f3d-03f8-44e4-becb-250de76f00f6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.407724 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a6f3d-03f8-44e4-becb-250de76f00f6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.411669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fce78-fe49-4b07-8b31-561d54ca1eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.412672 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fce78-fe49-4b07-8b31-561d54ca1eb0 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.413593 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fce78-fe49-4b07-8b31-561d54ca1eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.414018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fce78-fe49-4b07-8b31-561d54ca1eb0 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.414712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4fce78-fe49-4b07-8b31-561d54ca1eb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:48.415013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4fce78-fe49-4b07-8b31-561d54ca1eb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.416455 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4fce78-fe49-4b07-8b31-561d54ca1eb0 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-166a6f3d-03f8-44e4-becb-250de76f00f6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:48.422266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:48.423226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 309/585] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:48.429478 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a6f3d-03f8-44e4-becb-250de76f00f6 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.433558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 277/586] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:48.439048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fce78-fe49-4b07-8b31-561d54ca1eb0 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.536227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a6f3d-03f8-44e4-becb-250de76f00f6 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.538734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:48.542314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fce78-fe49-4b07-8b31-561d54ca1eb0 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.575170 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa0da8-7072-401a-ac3c-941c685b8826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.576087 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa0da8-7072-401a-ac3c-941c685b8826 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.576498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa0da8-7072-401a-ac3c-941c685b8826 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.576926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa0da8-7072-401a-ac3c-941c685b8826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.579339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd8509-f40a-40a9-8feb-daa8f59a0e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.579971 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8509-f40a-40a9-8feb-daa8f59a0e9e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.580255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8509-f40a-40a9-8feb-daa8f59a0e9e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.580513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8509-f40a-40a9-8feb-daa8f59a0e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.589917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa0da8-7072-401a-ac3c-941c685b8826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.590756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd8509-f40a-40a9-8feb-daa8f59a0e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.591796 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa0da8-7072-401a-ac3c-941c685b8826 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.592225 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd8509-f40a-40a9-8feb-daa8f59a0e9e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-befa0da8-7072-401a-ac3c-941c685b8826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:48.607257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:48.608227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:48.609125 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:48.609125 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:48.609125 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-78fd8509-f40a-40a9-8feb-daa8f59a0e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:48.609303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:48.610218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:48.611093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:48.611093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:48.611093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:48.611295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 310/587] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:48.611295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa0da8-7072-401a-ac3c-941c685b8826 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.614116 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 278/588] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:48.614388 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd8509-f40a-40a9-8feb-daa8f59a0e9e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.647259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:48.718600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd8509-f40a-40a9-8feb-daa8f59a0e9e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.719093 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa0da8-7072-401a-ac3c-941c685b8826 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.755767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf137a-094f-4031-933f-83ea0fca7ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.755997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d863953-71d6-473f-a516-c9d843033ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.756428 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf137a-094f-4031-933f-83ea0fca7ed6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.756507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d863953-71d6-473f-a516-c9d843033ef4 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.756754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf137a-094f-4031-933f-83ea0fca7ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.756864 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d863953-71d6-473f-a516-c9d843033ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.756939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf137a-094f-4031-933f-83ea0fca7ed6 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.757022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d863953-71d6-473f-a516-c9d843033ef4 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/0bb80229c3234d5f9599b16870c69934/roles/0ee8ef3183064d76a7e90b3ae6c5376d` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.757605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf137a-094f-4031-933f-83ea0fca7ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:48.757955 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf137a-094f-4031-933f-83ea0fca7ed6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.759378 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf137a-094f-4031-933f-83ea0fca7ed6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.772211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d863953-71d6-473f-a516-c9d843033ef4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=0bb80229c3234d5f9599b16870c69934, role_id=0ee8ef3183064d76a7e90b3ae6c5376d)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.773644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d863953-71d6-473f-a516-c9d843033ef4 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.777419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 311/589] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:48.778269 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf137a-094f-4031-933f-83ea0fca7ed6 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.788057 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 279/590] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:36:48 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/0bb80229c3234d5f9599b16870c69934/roles/0ee8ef3183064d76a7e90b3ae6c5376d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:48.789070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d863953-71d6-473f-a516-c9d843033ef4 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.791651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:48.885701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:48.900718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf137a-094f-4031-933f-83ea0fca7ed6 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.903310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d863953-71d6-473f-a516-c9d843033ef4 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:48.937934 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5d49d-3cc1-4adc-87e4-9516bc34ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.938862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5d49d-3cc1-4adc-87e4-9516bc34ff49 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.939463 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5d49d-3cc1-4adc-87e4-9516bc34ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.939921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5d49d-3cc1-4adc-87e4-9516bc34ff49 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.940675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5d49d-3cc1-4adc-87e4-9516bc34ff49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:48.940977 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5d49d-3cc1-4adc-87e4-9516bc34ff49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.942394 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5d49d-3cc1-4adc-87e4-9516bc34ff49 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.945145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00a785-a473-4272-a575-8c12f9ba601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:48.945842 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00a785-a473-4272-a575-8c12f9ba601f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.946118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00a785-a473-4272-a575-8c12f9ba601f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.946387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00a785-a473-4272-a575-8c12f9ba601f None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.947032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00a785-a473-4272-a575-8c12f9ba601f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:48.947505 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00a785-a473-4272-a575-8c12f9ba601f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:48.949562 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00a785-a473-4272-a575-8c12f9ba601f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:48.955996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 312/591] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:48.957359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed6e7e-49d5-43dd-abae-7815e9678755 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:48.957743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed6e7e-49d5-43dd-abae-7815e9678755 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:48.958007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed6e7e-49d5-43dd-abae-7815e9678755 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:48.959868 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 313/592] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:48.960213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed6e7e-49d5-43dd-abae-7815e9678755 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:48.965381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 280/593] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:48.966264 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00a785-a473-4272-a575-8c12f9ba601f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.028342 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:49.067876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00a785-a473-4272-a575-8c12f9ba601f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.076247 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed6e7e-49d5-43dd-abae-7815e9678755 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.102061 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e6c52-0d2a-4386-a088-4669a7ab0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.102969 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e6c52-0d2a-4386-a088-4669a7ab0414 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.103277 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e6c52-0d2a-4386-a088-4669a7ab0414 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.103548 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e6c52-0d2a-4386-a088-4669a7ab0414 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.110657 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e6c52-0d2a-4386-a088-4669a7ab0414 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:49.112111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e6c52-0d2a-4386-a088-4669a7ab0414 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:49.113809 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01925223-1df8-4350-8ce2-aac7ac9c1fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.114633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01925223-1df8-4350-8ce2-aac7ac9c1fc9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.114905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01925223-1df8-4350-8ce2-aac7ac9c1fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.115189 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01925223-1df8-4350-8ce2-aac7ac9c1fc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-106e6c52-0d2a-4386-a088-4669a7ab0414 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:49.123645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:49.124552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:49.125451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:49.125451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:49.125451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:49.125451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:49.125451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:49.125451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:49.125451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:49.125451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:49.125451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 281/594] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:49.125946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b9844-21e8-4c58-a06f-b91a4668115d None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.126852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b9844-21e8-4c58-a06f-b91a4668115d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.126852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b9844-21e8-4c58-a06f-b91a4668115d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.131518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01925223-1df8-4350-8ce2-aac7ac9c1fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:49.132979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01925223-1df8-4350-8ce2-aac7ac9c1fc9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:49.141534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:49.145697 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-040b9844-21e8-4c58-a06f-b91a4668115d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:49.146883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-040b9844-21e8-4c58-a06f-b91a4668115d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-01925223-1df8-4350-8ce2-aac7ac9c1fc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:49.149672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:49.150622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:49.151536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:49.151536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:49.151536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:49.155435 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 314/595] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:49.156225 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01925223-1df8-4350-8ce2-aac7ac9c1fc9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.220111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 282/596] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:49.221027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b9844-21e8-4c58-a06f-b91a4668115d None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.276838 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01925223-1df8-4350-8ce2-aac7ac9c1fc9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.286650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:49.312026 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e89bcd-97f6-4dc8-9273-0db1b56d3c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.312663 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e89bcd-97f6-4dc8-9273-0db1b56d3c4d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.312937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e89bcd-97f6-4dc8-9273-0db1b56d3c4d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.313257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e89bcd-97f6-4dc8-9273-0db1b56d3c4d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.321437 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e89bcd-97f6-4dc8-9273-0db1b56d3c4d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:49.323168 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e89bcd-97f6-4dc8-9273-0db1b56d3c4d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-73e89bcd-97f6-4dc8-9273-0db1b56d3c4d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:49.335048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:49.336001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:49.336902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 315/597] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:49.337493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e89bcd-97f6-4dc8-9273-0db1b56d3c4d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.348051 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b9844-21e8-4c58-a06f-b91a4668115d None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.383310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe79895-4dee-43c6-bf80-fe29c88f2228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.384253 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe79895-4dee-43c6-bf80-fe29c88f2228 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.384717 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe79895-4dee-43c6-bf80-fe29c88f2228 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.385273 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe79895-4dee-43c6-bf80-fe29c88f2228 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.386150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe79895-4dee-43c6-bf80-fe29c88f2228 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:49.386653 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe79895-4dee-43c6-bf80-fe29c88f2228 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:49.388381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe79895-4dee-43c6-bf80-fe29c88f2228 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:49.399459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:49.401490 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 283/598] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:49.402667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe79895-4dee-43c6-bf80-fe29c88f2228 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.446560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e89bcd-97f6-4dc8-9273-0db1b56d3c4d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.482349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b66818-fb33-46c1-886f-dd1e379a5c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.482718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b66818-fb33-46c1-886f-dd1e379a5c87 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.482999 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b66818-fb33-46c1-886f-dd1e379a5c87 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.483298 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b66818-fb33-46c1-886f-dd1e379a5c87 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.484000 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b66818-fb33-46c1-886f-dd1e379a5c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:49.484300 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b66818-fb33-46c1-886f-dd1e379a5c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:49.485715 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b66818-fb33-46c1-886f-dd1e379a5c87 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:49.498443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 316/599] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:49.499312 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b66818-fb33-46c1-886f-dd1e379a5c87 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.509439 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe79895-4dee-43c6-bf80-fe29c88f2228 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.522659 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:49.543669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a16766-5b09-4e3c-96fd-0623f474e458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.544192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a16766-5b09-4e3c-96fd-0623f474e458 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.544460 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a16766-5b09-4e3c-96fd-0623f474e458 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.544723 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a16766-5b09-4e3c-96fd-0623f474e458 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.545442 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a16766-5b09-4e3c-96fd-0623f474e458 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:49.545737 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a16766-5b09-4e3c-96fd-0623f474e458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:49.547164 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a16766-5b09-4e3c-96fd-0623f474e458 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:49.564125 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 284/600] 10.222.0.16 () {56 vars in 1139 bytes} [Tue Aug 30 11:36:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:49.564896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a16766-5b09-4e3c-96fd-0623f474e458 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.606202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b66818-fb33-46c1-886f-dd1e379a5c87 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.629346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:49.645040 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28c305-0636-4e92-88a6-b079a51739f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.645839 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28c305-0636-4e92-88a6-b079a51739f3 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.646181 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28c305-0636-4e92-88a6-b079a51739f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.646517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28c305-0636-4e92-88a6-b079a51739f3 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/efbe019e4d4646428bed474cef4009b8/roles/0ee8ef3183064d76a7e90b3ae6c5376d` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.660904 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28c305-0636-4e92-88a6-b079a51739f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=efbe019e4d4646428bed474cef4009b8, role_id=0ee8ef3183064d76a7e90b3ae6c5376d)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:49.662355 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28c305-0636-4e92-88a6-b079a51739f3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:49.673051 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a16766-5b09-4e3c-96fd-0623f474e458 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.683148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 317/601] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:36:49 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/efbe019e4d4646428bed474cef4009b8/roles/0ee8ef3183064d76a7e90b3ae6c5376d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:49.683148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28c305-0636-4e92-88a6-b079a51739f3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.713682 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc1f81-4c29-4ffc-9cf6-ee9d89923f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.714417 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc1f81-4c29-4ffc-9cf6-ee9d89923f0d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.714753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc1f81-4c29-4ffc-9cf6-ee9d89923f0d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.715082 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc1f81-4c29-4ffc-9cf6-ee9d89923f0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.727603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc1f81-4c29-4ffc-9cf6-ee9d89923f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-f3fc1f81-4c29-4ffc-9cf6-ee9d89923f0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:49.732788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:49.733741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:49.734741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 285/602] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:49.735657 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc1f81-4c29-4ffc-9cf6-ee9d89923f0d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.771509 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:49.817531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28c305-0636-4e92-88a6-b079a51739f3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.855628 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb9cae-1c36-4f2a-a2d2-68c42bde3da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.856614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb9cae-1c36-4f2a-a2d2-68c42bde3da0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.857092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb9cae-1c36-4f2a-a2d2-68c42bde3da0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.857360 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb9cae-1c36-4f2a-a2d2-68c42bde3da0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.859705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc1f81-4c29-4ffc-9cf6-ee9d89923f0d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:49.867160 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb9cae-1c36-4f2a-a2d2-68c42bde3da0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:49.868611 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb9cae-1c36-4f2a-a2d2-68c42bde3da0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-b9eb9cae-1c36-4f2a-a2d2-68c42bde3da0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:49.883878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:49.884871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:49.885752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:49.885752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:49.885752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:49.885752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:49.885752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:49.885752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:49.885752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:49.885752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:49.885752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 318/603] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:49.886669 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb9cae-1c36-4f2a-a2d2-68c42bde3da0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:49.907881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4f7d9-16f2-4a41-9d2b-4246e6cd8267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:49.908791 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4f7d9-16f2-4a41-9d2b-4246e6cd8267 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:49.909222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4f7d9-16f2-4a41-9d2b-4246e6cd8267 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:49.909728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4f7d9-16f2-4a41-9d2b-4246e6cd8267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:49.921430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4f7d9-16f2-4a41-9d2b-4246e6cd8267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:49.923226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4f7d9-16f2-4a41-9d2b-4246e6cd8267 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:49.923973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-6be4f7d9-16f2-4a41-9d2b-4246e6cd8267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:49.941755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:49.942972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:49.944162 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:49.944162 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:49.944162 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:49.945696 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 286/604] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:49.946687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4f7d9-16f2-4a41-9d2b-4246e6cd8267 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:50.015475 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb9cae-1c36-4f2a-a2d2-68c42bde3da0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.049373 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:50.055209 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be8b4a-5435-44e6-88a2-54f92ba71576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.055832 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be8b4a-5435-44e6-88a2-54f92ba71576 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.056108 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be8b4a-5435-44e6-88a2-54f92ba71576 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.056383 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be8b4a-5435-44e6-88a2-54f92ba71576 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.057053 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be8b4a-5435-44e6-88a2-54f92ba71576 None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:50.057354 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be8b4a-5435-44e6-88a2-54f92ba71576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.058760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be8b4a-5435-44e6-88a2-54f92ba71576 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.070803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 319/605] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:50.071759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4f7d9-16f2-4a41-9d2b-4246e6cd8267 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.072959 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be8b4a-5435-44e6-88a2-54f92ba71576 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:50.121204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335541f0-2348-4bc7-a5ab-0f9f59621dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.122236 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335541f0-2348-4bc7-a5ab-0f9f59621dab None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.122708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335541f0-2348-4bc7-a5ab-0f9f59621dab None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.123257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335541f0-2348-4bc7-a5ab-0f9f59621dab None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.124218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335541f0-2348-4bc7-a5ab-0f9f59621dab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:50.124712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335541f0-2348-4bc7-a5ab-0f9f59621dab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.126387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335541f0-2348-4bc7-a5ab-0f9f59621dab None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.145029 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 287/606] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:50.146243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335541f0-2348-4bc7-a5ab-0f9f59621dab None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:50.185371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:50.196933 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be8b4a-5435-44e6-88a2-54f92ba71576 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.243157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8024bf-cd9b-4a00-9dce-1e77adcdc1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.243157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8024bf-cd9b-4a00-9dce-1e77adcdc1af None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.243157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8024bf-cd9b-4a00-9dce-1e77adcdc1af None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.243157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8024bf-cd9b-4a00-9dce-1e77adcdc1af None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.243609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8024bf-cd9b-4a00-9dce-1e77adcdc1af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:50.243609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8024bf-cd9b-4a00-9dce-1e77adcdc1af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.243609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8024bf-cd9b-4a00-9dce-1e77adcdc1af None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.256639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335541f0-2348-4bc7-a5ab-0f9f59621dab None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.260399 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 320/607] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:50.261384 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8024bf-cd9b-4a00-9dce-1e77adcdc1af None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:50.296553 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e18e6-8489-49bf-8574-30a090a7e7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.297188 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e18e6-8489-49bf-8574-30a090a7e7f7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.297454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e18e6-8489-49bf-8574-30a090a7e7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.297806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e18e6-8489-49bf-8574-30a090a7e7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.308252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e18e6-8489-49bf-8574-30a090a7e7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.309874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e18e6-8489-49bf-8574-30a090a7e7f7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.319431 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-b20e18e6-8489-49bf-8574-30a090a7e7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:50.327314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:50.328271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:50.329478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:50.329478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:50.329478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:50.333628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 288/608] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:50.334310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e18e6-8489-49bf-8574-30a090a7e7f7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:50.380778 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8024bf-cd9b-4a00-9dce-1e77adcdc1af None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.415772 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a815c-b62b-4153-9acc-2558d34ec730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.416314 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a815c-b62b-4153-9acc-2558d34ec730 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.416970 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a815c-b62b-4153-9acc-2558d34ec730 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.417235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a815c-b62b-4153-9acc-2558d34ec730 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/d3250e8f23c64f26b4e862090f7f2e18/roles/0ee8ef3183064d76a7e90b3ae6c5376d` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.432408 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a815c-b62b-4153-9acc-2558d34ec730 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=d3250e8f23c64f26b4e862090f7f2e18, role_id=0ee8ef3183064d76a7e90b3ae6c5376d)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.433862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a815c-b62b-4153-9acc-2558d34ec730 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.451619 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 321/609] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:36:50 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/d3250e8f23c64f26b4e862090f7f2e18/roles/0ee8ef3183064d76a7e90b3ae6c5376d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:50.452815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a815c-b62b-4153-9acc-2558d34ec730 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:50.456177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:50.465764 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e18e6-8489-49bf-8574-30a090a7e7f7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.505105 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7996bd-da61-4bd2-b011-000d2fa4fa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.506105 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7996bd-da61-4bd2-b011-000d2fa4fa75 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.506556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7996bd-da61-4bd2-b011-000d2fa4fa75 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.507020 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7996bd-da61-4bd2-b011-000d2fa4fa75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.515042 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7996bd-da61-4bd2-b011-000d2fa4fa75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.516924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7996bd-da61-4bd2-b011-000d2fa4fa75 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-8d7996bd-da61-4bd2-b011-000d2fa4fa75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:50.530031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:50.530979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:50.531888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:50.531888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:50.531888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:50.531888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:50.531888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:50.531888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:50.531888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:50.531888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:50.534430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 289/610] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:50.540929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7996bd-da61-4bd2-b011-000d2fa4fa75 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:50.615714 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a815c-b62b-4153-9acc-2558d34ec730 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.660333 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:50.671712 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ccd0c-cfb7-4ef4-b1af-864c553abfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.672435 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ccd0c-cfb7-4ef4-b1af-864c553abfcd None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.672711 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ccd0c-cfb7-4ef4-b1af-864c553abfcd None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.672974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ccd0c-cfb7-4ef4-b1af-864c553abfcd None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.673623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ccd0c-cfb7-4ef4-b1af-864c553abfcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:50.673922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ccd0c-cfb7-4ef4-b1af-864c553abfcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.675326 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ccd0c-cfb7-4ef4-b1af-864c553abfcd None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.689184 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 322/611] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:50.695640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ccd0c-cfb7-4ef4-b1af-864c553abfcd None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:50.727806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7996bd-da61-4bd2-b011-000d2fa4fa75 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.773370 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:50.774708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182e32b-d278-44c9-b5b5-2f3497c7d9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.775411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e32b-d278-44c9-b5b5-2f3497c7d9a1 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.775729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e32b-d278-44c9-b5b5-2f3497c7d9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.776149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e32b-d278-44c9-b5b5-2f3497c7d9a1 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.776866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182e32b-d278-44c9-b5b5-2f3497c7d9a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:50.777682 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182e32b-d278-44c9-b5b5-2f3497c7d9a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.779535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182e32b-d278-44c9-b5b5-2f3497c7d9a1 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.796096 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 290/612] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:50.797819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588c1ae-117a-48aa-bf79-7e534e81c90d None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.798107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588c1ae-117a-48aa-bf79-7e534e81c90d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.798381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588c1ae-117a-48aa-bf79-7e534e81c90d None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.799968 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 291/613] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:50.802446 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588c1ae-117a-48aa-bf79-7e534e81c90d None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:50.818960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ccd0c-cfb7-4ef4-b1af-864c553abfcd None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.855925 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33348b3a-cca6-4fce-9292-375a30c17619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.856570 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33348b3a-cca6-4fce-9292-375a30c17619 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.856848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33348b3a-cca6-4fce-9292-375a30c17619 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.857148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33348b3a-cca6-4fce-9292-375a30c17619 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.865783 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33348b3a-cca6-4fce-9292-375a30c17619 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.867336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33348b3a-cca6-4fce-9292-375a30c17619 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-33348b3a-cca6-4fce-9292-375a30c17619 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:50.880023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:50.880969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:50.881901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 323/614] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:36:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:50.882986 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3a32-4f03-4b49-bda7-637197454d42 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.882986 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3a32-4f03-4b49-bda7-637197454d42 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.883320 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3a32-4f03-4b49-bda7-637197454d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.902746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-97fc3a32-4f03-4b49-bda7-637197454d42 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:50.904086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-97fc3a32-4f03-4b49-bda7-637197454d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:50.910145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588c1ae-117a-48aa-bf79-7e534e81c90d None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:50.928739 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:50.950646 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcda0b-447d-4f42-b1ef-6bccc4e49176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:50.951606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcda0b-447d-4f42-b1ef-6bccc4e49176 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:50.952037 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcda0b-447d-4f42-b1ef-6bccc4e49176 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:50.952375 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcda0b-447d-4f42-b1ef-6bccc4e49176 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/56bc4fdd14d147ef86b96786da2855f5/roles/0ee8ef3183064d76a7e90b3ae6c5376d` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:50.968971 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcda0b-447d-4f42-b1ef-6bccc4e49176 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=56bc4fdd14d147ef86b96786da2855f5, role_id=0ee8ef3183064d76a7e90b3ae6c5376d)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:50.970508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcda0b-447d-4f42-b1ef-6bccc4e49176 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:50.984968 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 292/616] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:36:50 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/56bc4fdd14d147ef86b96786da2855f5/roles/0ee8ef3183064d76a7e90b3ae6c5376d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 11:36:50.985387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 324/616] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:50.986134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcda0b-447d-4f42-b1ef-6bccc4e49176 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:51.103770 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcda0b-447d-4f42-b1ef-6bccc4e49176 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:51.143447 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b9bfd-7d28-47b8-8435-6fd66d3c4b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:51.143987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b9bfd-7d28-47b8-8435-6fd66d3c4b75 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:51.144264 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b9bfd-7d28-47b8-8435-6fd66d3c4b75 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:51.144531 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b9bfd-7d28-47b8-8435-6fd66d3c4b75 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:51.145349 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b9bfd-7d28-47b8-8435-6fd66d3c4b75 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:51.145652 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b9bfd-7d28-47b8-8435-6fd66d3c4b75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:51.147171 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b9bfd-7d28-47b8-8435-6fd66d3c4b75 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:51.167092 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 293/617] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Aug 30 11:36:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:51.172626 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3a32-4f03-4b49-bda7-637197454d42 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:51.233647 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:51.292248 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3a32-4f03-4b49-bda7-637197454d42 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:51.320518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b9bfd-7d28-47b8-8435-6fd66d3c4b75 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:51.327837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4835242-1857-46e7-b9c4-47532979b49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:51.329051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4835242-1857-46e7-b9c4-47532979b49a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:51.329557 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4835242-1857-46e7-b9c4-47532979b49a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:51.330011 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4835242-1857-46e7-b9c4-47532979b49a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:51.341008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4835242-1857-46e7-b9c4-47532979b49a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:51.342800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4835242-1857-46e7-b9c4-47532979b49a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:51.355478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-a4835242-1857-46e7-b9c4-47532979b49a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:51.358232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:51.359277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:51.360215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:51.360215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:51.360215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:51.360215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 325/618] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:51.365382 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4835242-1857-46e7-b9c4-47532979b49a None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:51.481496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4835242-1857-46e7-b9c4-47532979b49a None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:51.483981 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:51.517433 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa44d3-a5e1-4ed6-81d8-18d8a33b958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:51.518042 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa44d3-a5e1-4ed6-81d8-18d8a33b958f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:51.518311 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa44d3-a5e1-4ed6-81d8-18d8a33b958f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:51.518573 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa44d3-a5e1-4ed6-81d8-18d8a33b958f None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:51.518821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b9bfd-7d28-47b8-8435-6fd66d3c4b75 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:51.519237 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa44d3-a5e1-4ed6-81d8-18d8a33b958f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:51.519536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa44d3-a5e1-4ed6-81d8-18d8a33b958f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:51.521037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa44d3-a5e1-4ed6-81d8-18d8a33b958f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:51.533572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 326/619] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:51.541620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa44d3-a5e1-4ed6-81d8-18d8a33b958f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:51.609196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e9290-4fa0-42c1-beea-fd3333071d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:51.609832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e9290-4fa0-42c1-beea-fd3333071d64 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:51.610102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e9290-4fa0-42c1-beea-fd3333071d64 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:51.610360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e9290-4fa0-42c1-beea-fd3333071d64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:51.644016 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e9290-4fa0-42c1-beea-fd3333071d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:51.647437 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa44d3-a5e1-4ed6-81d8-18d8a33b958f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-a18e9290-4fa0-42c1-beea-fd3333071d64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:51.648881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:51.649961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:51.650806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 294/620] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:51.655773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e9290-4fa0-42c1-beea-fd3333071d64 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:51.699905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935dc4-d8b0-4022-bf9f-5c711da1e44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:51.700454 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935dc4-d8b0-4022-bf9f-5c711da1e44a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:51.700725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935dc4-d8b0-4022-bf9f-5c711da1e44a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:51.700987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935dc4-d8b0-4022-bf9f-5c711da1e44a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:51.714385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5935dc4-d8b0-4022-bf9f-5c711da1e44a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:51.715875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5935dc4-d8b0-4022-bf9f-5c711da1e44a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-a5935dc4-d8b0-4022-bf9f-5c711da1e44a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:51.740443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:51.741419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:51.742296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:51.742296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:51.742296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:51.742296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:51.742296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:51.742296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:51.742296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:51.742296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:51.742754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 327/621] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:51.749328 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935dc4-d8b0-4022-bf9f-5c711da1e44a None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:51.796339 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:51.817436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:51.853222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e9290-4fa0-42c1-beea-fd3333071d64 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:51.898472 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935dc4-d8b0-4022-bf9f-5c711da1e44a None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:51.900864 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175816b-421d-4d6e-bd32-0325b9d886f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:51.902524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175816b-421d-4d6e-bd32-0325b9d886f4 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:51.902867 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175816b-421d-4d6e-bd32-0325b9d886f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:51.903238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175816b-421d-4d6e-bd32-0325b9d886f4 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:51.903981 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175816b-421d-4d6e-bd32-0325b9d886f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:51.904343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175816b-421d-4d6e-bd32-0325b9d886f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:51.905826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175816b-421d-4d6e-bd32-0325b9d886f4 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:51.919229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 295/622] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:51.920485 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d4aff-d443-4bd7-b862-35798345dde7 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:51.920763 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d4aff-d443-4bd7-b862-35798345dde7 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:51.921026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d4aff-d443-4bd7-b862-35798345dde7 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:51.926049 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 296/623] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:51.926049 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d4aff-d443-4bd7-b862-35798345dde7 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:51.943243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68c30d-6a99-4344-b779-cc0d3cd9b1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:51.947139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c30d-6a99-4344-b779-cc0d3cd9b1f3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:51.947139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c30d-6a99-4344-b779-cc0d3cd9b1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:51.947139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c30d-6a99-4344-b779-cc0d3cd9b1f3 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:51.947139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68c30d-6a99-4344-b779-cc0d3cd9b1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:51.947139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68c30d-6a99-4344-b779-cc0d3cd9b1f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:51.947811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68c30d-6a99-4344-b779-cc0d3cd9b1f3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:51.967267 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 328/624] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:51.971137 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7354f74-73fd-49a7-8894-82bb273229b2 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:51.971137 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7354f74-73fd-49a7-8894-82bb273229b2 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:51.971137 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7354f74-73fd-49a7-8894-82bb273229b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:51.989310 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-b7354f74-73fd-49a7-8894-82bb273229b2 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:51.990820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-b7354f74-73fd-49a7-8894-82bb273229b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:52.040293 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d4aff-d443-4bd7-b862-35798345dde7 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:52.062406 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 329/625] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:52.064231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7354f74-73fd-49a7-8894-82bb273229b2 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:52.075829 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e7e6a-55f8-4307-9922-025429fb0ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:52.076628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e7e6a-55f8-4307-9922-025429fb0ed2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:52.076897 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e7e6a-55f8-4307-9922-025429fb0ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:52.077161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e7e6a-55f8-4307-9922-025429fb0ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:52.083947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:52.092887 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e7e6a-55f8-4307-9922-025429fb0ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:52.094592 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e7e6a-55f8-4307-9922-025429fb0ed2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:52.097559 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-155e7e6a-55f8-4307-9922-025429fb0ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:52.127459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:52.129774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:52.131901 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:52.131901 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:52.131901 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:52.131901 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 297/626] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:52.143662 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e7e6a-55f8-4307-9922-025429fb0ed2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:52.178702 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7354f74-73fd-49a7-8894-82bb273229b2 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:52.215081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459fda8-0fbe-4ec3-80dc-b6cedc65b4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:52.215777 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459fda8-0fbe-4ec3-80dc-b6cedc65b4bf None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:52.216050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459fda8-0fbe-4ec3-80dc-b6cedc65b4bf None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:52.216415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459fda8-0fbe-4ec3-80dc-b6cedc65b4bf None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/8bbb8fb036684305bc00290c519939ec/roles/0ee8ef3183064d76a7e90b3ae6c5376d` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:52.233067 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459fda8-0fbe-4ec3-80dc-b6cedc65b4bf None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=8bbb8fb036684305bc00290c519939ec, role_id=0ee8ef3183064d76a7e90b3ae6c5376d)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:52.234760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459fda8-0fbe-4ec3-80dc-b6cedc65b4bf None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:52.249605 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e7e6a-55f8-4307-9922-025429fb0ed2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:52.257094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 330/627] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:36:52 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/8bbb8fb036684305bc00290c519939ec/roles/0ee8ef3183064d76a7e90b3ae6c5376d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:36:52.312877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8171f53-9ea4-4bc9-bb89-10d3719b3b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:52.313548 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171f53-9ea4-4bc9-bb89-10d3719b3b7d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:52.313867 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171f53-9ea4-4bc9-bb89-10d3719b3b7d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:52.314135 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171f53-9ea4-4bc9-bb89-10d3719b3b7d None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:52.314842 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171f53-9ea4-4bc9-bb89-10d3719b3b7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:52.315166 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171f53-9ea4-4bc9-bb89-10d3719b3b7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:52.316648 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171f53-9ea4-4bc9-bb89-10d3719b3b7d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:52.334716 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 298/628] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:52.340672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459fda8-0fbe-4ec3-80dc-b6cedc65b4bf None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:52.428128 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:52.447159 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8171f53-9ea4-4bc9-bb89-10d3719b3b7d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:52.474114 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459fda8-0fbe-4ec3-80dc-b6cedc65b4bf None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:52.515498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d84b7-dbe5-4784-8187-6c582ad98a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:52.516482 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d84b7-dbe5-4784-8187-6c582ad98a32 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:52.516958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d84b7-dbe5-4784-8187-6c582ad98a32 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:52.517444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d84b7-dbe5-4784-8187-6c582ad98a32 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:52.525398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d84b7-dbe5-4784-8187-6c582ad98a32 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:52.527133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d84b7-dbe5-4784-8187-6c582ad98a32 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:52.535750 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-257d84b7-dbe5-4784-8187-6c582ad98a32 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:52.539742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:52.540719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:36:52.541664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:52.544267 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 331/629] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Aug 30 11:36:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:52.554496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d84b7-dbe5-4784-8187-6c582ad98a32 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:52.600759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8171f53-9ea4-4bc9-bb89-10d3719b3b7d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:52.663019 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8ab4f-b91c-4e28-ac99-b047515c4761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:52.664768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8ab4f-b91c-4e28-ac99-b047515c4761 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:52.665815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8ab4f-b91c-4e28-ac99-b047515c4761 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:52.666458 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8ab4f-b91c-4e28-ac99-b047515c4761 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:52.689217 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8ab4f-b91c-4e28-ac99-b047515c4761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-28f8ab4f-b91c-4e28-ac99-b047515c4761 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:52.694885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:52.695855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:52.696711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:52.699387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 299/630] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:52.706097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8ab4f-b91c-4e28-ac99-b047515c4761 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:52.747886 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d84b7-dbe5-4784-8187-6c582ad98a32 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:52.798889 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:52.818722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e6619-663a-4c47-bc95-00ca9cfcce35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:52.819731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e6619-663a-4c47-bc95-00ca9cfcce35 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:52.820604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e6619-663a-4c47-bc95-00ca9cfcce35 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:52.821371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e6619-663a-4c47-bc95-00ca9cfcce35 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:52.822567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e6619-663a-4c47-bc95-00ca9cfcce35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:52.823213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e6619-663a-4c47-bc95-00ca9cfcce35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:52.825260 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e6619-663a-4c47-bc95-00ca9cfcce35 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:52.860980 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 332/631] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Aug 30 11:36:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:36:52.865151 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e6619-663a-4c47-bc95-00ca9cfcce35 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:52.905591 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8ab4f-b91c-4e28-ac99-b047515c4761 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:52.929219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:52.953535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa683fb5-e771-4bba-b8b1-2bdf20a3dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:52.954739 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa683fb5-e771-4bba-b8b1-2bdf20a3dce9 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:52.955021 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa683fb5-e771-4bba-b8b1-2bdf20a3dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:52.955330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa683fb5-e771-4bba-b8b1-2bdf20a3dce9 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:52.956177 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa683fb5-e771-4bba-b8b1-2bdf20a3dce9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:52.956776 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa683fb5-e771-4bba-b8b1-2bdf20a3dce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:52.958485 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa683fb5-e771-4bba-b8b1-2bdf20a3dce9 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:52.969216 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 300/632] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:52.970517 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa7cc-23d9-427f-a534-55ea2375b667 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:52.970801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa7cc-23d9-427f-a534-55ea2375b667 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:52.971072 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa7cc-23d9-427f-a534-55ea2375b667 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:52.972530 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 301/633] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:52.974503 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aa7cc-23d9-427f-a534-55ea2375b667 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.003695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e6619-663a-4c47-bc95-00ca9cfcce35 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:53.039792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4520e-f084-4755-aa29-69834dc6bc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:53.040671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4520e-f084-4755-aa29-69834dc6bc6d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.041042 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4520e-f084-4755-aa29-69834dc6bc6d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.041402 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4520e-f084-4755-aa29-69834dc6bc6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.097494 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aa7cc-23d9-427f-a534-55ea2375b667 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:53.103166 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:53.129135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4520e-f084-4755-aa29-69834dc6bc6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:53.131362 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4520e-f084-4755-aa29-69834dc6bc6d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:53.135109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15736b3c-d36f-400f-b974-ea005cb61a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:53.135860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15736b3c-d36f-400f-b974-ea005cb61a19 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.136267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15736b3c-d36f-400f-b974-ea005cb61a19 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.136623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15736b3c-d36f-400f-b974-ea005cb61a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.148505 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15736b3c-d36f-400f-b974-ea005cb61a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:53.149967 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15736b3c-d36f-400f-b974-ea005cb61a19 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-15736b3c-d36f-400f-b974-ea005cb61a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:53.169302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:53.170251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:53.171163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:53.171163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:53.171163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:53.171838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 302/634] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:53.173577 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc67a58-73c3-4ab1-9a43-89533d8dfc14 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.173577 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc67a58-73c3-4ab1-9a43-89533d8dfc14 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.173577 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc67a58-73c3-4ab1-9a43-89533d8dfc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.196035 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-afc67a58-73c3-4ab1-9a43-89533d8dfc14 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:53.197927 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-afc67a58-73c3-4ab1-9a43-89533d8dfc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:53.226800 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:53.246922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 333/635] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:53.246922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4520e-f084-4755-aa29-69834dc6bc6d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.275445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 303/636] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:53.275445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b1e6-5dbc-4d31-987b-78e0bf59d76d None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.275445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b1e6-5dbc-4d31-987b-78e0bf59d76d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.275819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b1e6-5dbc-4d31-987b-78e0bf59d76d None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.277341 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 304/637] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:53.278232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1b1e6-5dbc-4d31-987b-78e0bf59d76d None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.334621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:53.355828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4520e-f084-4755-aa29-69834dc6bc6d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:53.393314 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd5c3f-9729-4e05-bb61-4377db8e4659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:53.394237 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd5c3f-9729-4e05-bb61-4377db8e4659 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.394658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd5c3f-9729-4e05-bb61-4377db8e4659 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.395177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd5c3f-9729-4e05-bb61-4377db8e4659 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.396024 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd5c3f-9729-4e05-bb61-4377db8e4659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:53.396470 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd5c3f-9729-4e05-bb61-4377db8e4659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:53.398037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd5c3f-9729-4e05-bb61-4377db8e4659 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:53.406197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1b1e6-5dbc-4d31-987b-78e0bf59d76d None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:53.413622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 334/638] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:53.415417 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b602f1c-f117-4434-8c8b-e8bd15303d30 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.415906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b602f1c-f117-4434-8c8b-e8bd15303d30 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.416401 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b602f1c-f117-4434-8c8b-e8bd15303d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.441190 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44038439-5ace-4f21-92bb-231a4d3c401f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:53.441443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-0b602f1c-f117-4434-8c8b-e8bd15303d30 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:53.441830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44038439-5ace-4f21-92bb-231a4d3c401f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.442145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44038439-5ace-4f21-92bb-231a4d3c401f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.442410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44038439-5ace-4f21-92bb-231a4d3c401f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.443614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-0b602f1c-f117-4434-8c8b-e8bd15303d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:53.453018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44038439-5ace-4f21-92bb-231a4d3c401f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:53.454980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44038439-5ace-4f21-92bb-231a4d3c401f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-44038439-5ace-4f21-92bb-231a4d3c401f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:53.470370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:53.471434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:53.472380 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:53.472380 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:53.472380 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:53.476307 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 305/639] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:53.477040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44038439-5ace-4f21-92bb-231a4d3c401f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.493872 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:53.525471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 335/640] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:53.526750 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b602f1c-f117-4434-8c8b-e8bd15303d30 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.583255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44038439-5ace-4f21-92bb-231a4d3c401f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:53.617853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68762a0a-92e1-4790-adbd-5d819a911d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:53.618500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68762a0a-92e1-4790-adbd-5d819a911d8e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.618773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68762a0a-92e1-4790-adbd-5d819a911d8e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.619034 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68762a0a-92e1-4790-adbd-5d819a911d8e None admin] PATH_INFO: `/v3/users/glance` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.620069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:53.627267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68762a0a-92e1-4790-adbd-5d819a911d8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:53.629576 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68762a0a-92e1-4790-adbd-5d819a911d8e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-68762a0a-92e1-4790-adbd-5d819a911d8e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:53.641397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:53.642337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:53.643295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 306/641] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:53.647796 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68762a0a-92e1-4790-adbd-5d819a911d8e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.654369 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b602f1c-f117-4434-8c8b-e8bd15303d30 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:53.689308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc0c7f-0b0e-4b87-978c-33e12b1c57f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:53.690161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc0c7f-0b0e-4b87-978c-33e12b1c57f9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.690590 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc0c7f-0b0e-4b87-978c-33e12b1c57f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.691017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc0c7f-0b0e-4b87-978c-33e12b1c57f9 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.691972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc0c7f-0b0e-4b87-978c-33e12b1c57f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:53.692407 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc0c7f-0b0e-4b87-978c-33e12b1c57f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:53.694066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc0c7f-0b0e-4b87-978c-33e12b1c57f9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:53.706917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 336/642] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:53.712423 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc0c7f-0b0e-4b87-978c-33e12b1c57f9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.719998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:53.751957 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68762a0a-92e1-4790-adbd-5d819a911d8e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:53.786701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db06adc-145e-4ec0-bcf3-fa8089ad6590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:53.787356 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db06adc-145e-4ec0-bcf3-fa8089ad6590 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.787626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db06adc-145e-4ec0-bcf3-fa8089ad6590 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.787889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db06adc-145e-4ec0-bcf3-fa8089ad6590 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.798533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db06adc-145e-4ec0-bcf3-fa8089ad6590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-5db06adc-145e-4ec0-bcf3-fa8089ad6590 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:53.804148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:53.805169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:53.806046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:53.810291 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 307/643] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:53.811154 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db06adc-145e-4ec0-bcf3-fa8089ad6590 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.821436 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc0c7f-0b0e-4b87-978c-33e12b1c57f9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:53.852470 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:53.859523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dd163-e653-45a1-a664-678adc865ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:53.860462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dd163-e653-45a1-a664-678adc865ff7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.860906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dd163-e653-45a1-a664-678adc865ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.861434 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dd163-e653-45a1-a664-678adc865ff7 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.862382 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53dd163-e653-45a1-a664-678adc865ff7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:53.862863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53dd163-e653-45a1-a664-678adc865ff7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:53.864943 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53dd163-e653-45a1-a664-678adc865ff7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:53.884608 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 337/644] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Aug 30 11:36:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:53.885712 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dd163-e653-45a1-a664-678adc865ff7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.917757 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db06adc-145e-4ec0-bcf3-fa8089ad6590 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:53.950813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:53.956780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5397b2-43a4-4e58-a453-9196eed73007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:53.957411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5397b2-43a4-4e58-a453-9196eed73007 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:53.957710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5397b2-43a4-4e58-a453-9196eed73007 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:53.957977 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5397b2-43a4-4e58-a453-9196eed73007 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:53.965093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5397b2-43a4-4e58-a453-9196eed73007 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:53.966525 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5397b2-43a4-4e58-a453-9196eed73007 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-fa5397b2-43a4-4e58-a453-9196eed73007 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:53.978731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:53.979680 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:53.980747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:53.980747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:53.980747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:53.980747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:53.980747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:53.980747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:53.980747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:53.980747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:53.980747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 308/645] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:53.981246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5397b2-43a4-4e58-a453-9196eed73007 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:53.996857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dd163-e653-45a1-a664-678adc865ff7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.031838 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fc387-e69b-445b-81ea-5e6fe76d0036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.032458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fc387-e69b-445b-81ea-5e6fe76d0036 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.032709 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fc387-e69b-445b-81ea-5e6fe76d0036 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.032974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fc387-e69b-445b-81ea-5e6fe76d0036 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.033648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fc387-e69b-445b-81ea-5e6fe76d0036 None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:54.033916 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fc387-e69b-445b-81ea-5e6fe76d0036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:54.035355 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fc387-e69b-445b-81ea-5e6fe76d0036 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:54.048292 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 338/646] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:54.048942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fc387-e69b-445b-81ea-5e6fe76d0036 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.086664 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5397b2-43a4-4e58-a453-9196eed73007 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.103813 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:54.121746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d1eba-7ed3-4d3d-bfe5-baea55ff975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.122392 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d1eba-7ed3-4d3d-bfe5-baea55ff975c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.122671 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d1eba-7ed3-4d3d-bfe5-baea55ff975c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.123035 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d1eba-7ed3-4d3d-bfe5-baea55ff975c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.134385 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d1eba-7ed3-4d3d-bfe5-baea55ff975c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-529d1eba-7ed3-4d3d-bfe5-baea55ff975c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:54.139890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:54.141027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:54.142113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:54.143657 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 309/647] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:54.143657 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d1eba-7ed3-4d3d-bfe5-baea55ff975c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.160433 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fc387-e69b-445b-81ea-5e6fe76d0036 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.195257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f575ea4-5270-4c40-bae3-ac49dbe62073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.195924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f575ea4-5270-4c40-bae3-ac49dbe62073 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.196193 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f575ea4-5270-4c40-bae3-ac49dbe62073 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.196456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f575ea4-5270-4c40-bae3-ac49dbe62073 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.199391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:54.251579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d1eba-7ed3-4d3d-bfe5-baea55ff975c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.267546 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f575ea4-5270-4c40-bae3-ac49dbe62073 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:54.269018 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f575ea4-5270-4c40-bae3-ac49dbe62073 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:54.286394 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c783286-19ba-4213-9262-a0569fdbdd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.287259 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c783286-19ba-4213-9262-a0569fdbdd83 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.287539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c783286-19ba-4213-9262-a0569fdbdd83 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.287871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c783286-19ba-4213-9262-a0569fdbdd83 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.288605 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c783286-19ba-4213-9262-a0569fdbdd83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:54.288962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c783286-19ba-4213-9262-a0569fdbdd83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:54.290520 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c783286-19ba-4213-9262-a0569fdbdd83 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:54.303194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 310/648] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:54.304057 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c783286-19ba-4213-9262-a0569fdbdd83 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.325019 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:54.375522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 339/649] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:54.376362 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f575ea4-5270-4c40-bae3-ac49dbe62073 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.404563 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c783286-19ba-4213-9262-a0569fdbdd83 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.419317 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:54.440599 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c9b70-854b-4d74-bd47-2e4700691026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.441229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c9b70-854b-4d74-bd47-2e4700691026 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.441501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c9b70-854b-4d74-bd47-2e4700691026 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.441762 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c9b70-854b-4d74-bd47-2e4700691026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.451669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c9b70-854b-4d74-bd47-2e4700691026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:54.453106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c9b70-854b-4d74-bd47-2e4700691026 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-f48c9b70-854b-4d74-bd47-2e4700691026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:54.468429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:54.469542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:54.470684 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:54.470684 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:54.470684 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:54.470684 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 311/650] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:54.475004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c9b70-854b-4d74-bd47-2e4700691026 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.477187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f575ea4-5270-4c40-bae3-ac49dbe62073 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.513162 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a89718-80d9-4306-9fb7-e7e5173f5e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.513901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a89718-80d9-4306-9fb7-e7e5173f5e8c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.514285 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a89718-80d9-4306-9fb7-e7e5173f5e8c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.514626 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a89718-80d9-4306-9fb7-e7e5173f5e8c None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.515383 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a89718-80d9-4306-9fb7-e7e5173f5e8c None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:54.515742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a89718-80d9-4306-9fb7-e7e5173f5e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:54.517182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a89718-80d9-4306-9fb7-e7e5173f5e8c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:54.524958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 340/651] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:54.526136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8408cc-e7df-4911-bba0-e18aa47be471 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.526415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8408cc-e7df-4911-bba0-e18aa47be471 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.526686 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8408cc-e7df-4911-bba0-e18aa47be471 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.532078 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 341/652] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:54.532078 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8408cc-e7df-4911-bba0-e18aa47be471 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.544911 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:54.574687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c9b70-854b-4d74-bd47-2e4700691026 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.609725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cc512-51b8-4a34-957c-95f399c33fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.610350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cc512-51b8-4a34-957c-95f399c33fdb None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.610621 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cc512-51b8-4a34-957c-95f399c33fdb None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.610882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cc512-51b8-4a34-957c-95f399c33fdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.611626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cc512-51b8-4a34-957c-95f399c33fdb None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=0bb80229c3234d5f9599b16870c69934) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:54.611922 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cc512-51b8-4a34-957c-95f399c33fdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:54.613362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cc512-51b8-4a34-957c-95f399c33fdb None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:54.619074 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 312/653] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/role_assignments?user.id=0bb80229c3234d5f9599b16870c69934&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:54.619868 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cc512-51b8-4a34-957c-95f399c33fdb None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.632821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8408cc-e7df-4911-bba0-e18aa47be471 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.649695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:54.674390 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7a7f8-4567-474f-bddd-aff776519a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.674968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7a7f8-4567-474f-bddd-aff776519a85 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.675281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7a7f8-4567-474f-bddd-aff776519a85 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.675556 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7a7f8-4567-474f-bddd-aff776519a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.688689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7a7f8-4567-474f-bddd-aff776519a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:54.690292 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7a7f8-4567-474f-bddd-aff776519a85 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-bde7a7f8-4567-474f-bddd-aff776519a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:54.707023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:54.708290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:54.709352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:54.709352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:54.709352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:54.713184 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 342/654] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:54.714637 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7a7f8-4567-474f-bddd-aff776519a85 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.733332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cc512-51b8-4a34-957c-95f399c33fdb None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.772047 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fba704-7137-41b2-b388-aa59c7d2fc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.772562 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba704-7137-41b2-b388-aa59c7d2fc87 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.773119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba704-7137-41b2-b388-aa59c7d2fc87 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.773396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba704-7137-41b2-b388-aa59c7d2fc87 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.774041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fba704-7137-41b2-b388-aa59c7d2fc87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:54.774339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fba704-7137-41b2-b388-aa59c7d2fc87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:54.776252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fba704-7137-41b2-b388-aa59c7d2fc87 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:54.788870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:54.790401 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 313/655] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:54.791808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d6e4c-887e-4094-aeae-e2058b85e898 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.792081 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d6e4c-887e-4094-aeae-e2058b85e898 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.792348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d6e4c-887e-4094-aeae-e2058b85e898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.810915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-686d6e4c-887e-4094-aeae-e2058b85e898 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:54.812144 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-686d6e4c-887e-4094-aeae-e2058b85e898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:54.835382 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7a7f8-4567-474f-bddd-aff776519a85 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:54.869099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae133d-d3d5-4b00-8986-99f0d3c25f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:54.869732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae133d-d3d5-4b00-8986-99f0d3c25f59 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:54.870001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae133d-d3d5-4b00-8986-99f0d3c25f59 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:54.870262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae133d-d3d5-4b00-8986-99f0d3c25f59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:54.880858 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae133d-d3d5-4b00-8986-99f0d3c25f59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:54.882366 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae133d-d3d5-4b00-8986-99f0d3c25f59 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:54.888556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 314/656] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:54.889307 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d6e4c-887e-4094-aeae-e2058b85e898 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-94ae133d-d3d5-4b00-8986-99f0d3c25f59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:54.907946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:54.908905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:54.909792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:54.909792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:54.909792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:54.909792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 343/657] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:54.910443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae133d-d3d5-4b00-8986-99f0d3c25f59 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:54.913543 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:55.013860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d6e4c-887e-4094-aeae-e2058b85e898 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.029541 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae133d-d3d5-4b00-8986-99f0d3c25f59 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.047907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:55.056985 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a91412-e454-44c1-b6c3-5ad2ee968720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.057937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a91412-e454-44c1-b6c3-5ad2ee968720 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.058221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a91412-e454-44c1-b6c3-5ad2ee968720 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.058498 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a91412-e454-44c1-b6c3-5ad2ee968720 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.059443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a91412-e454-44c1-b6c3-5ad2ee968720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:55.059747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a91412-e454-44c1-b6c3-5ad2ee968720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.062815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a91412-e454-44c1-b6c3-5ad2ee968720 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.070200 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e08f7-6b24-456c-81ff-a28795ca70b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.071150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e08f7-6b24-456c-81ff-a28795ca70b9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.071855 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e08f7-6b24-456c-81ff-a28795ca70b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.072388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e08f7-6b24-456c-81ff-a28795ca70b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.076373 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 315/658] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:55.082917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e08f7-6b24-456c-81ff-a28795ca70b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.083085 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a91412-e454-44c1-b6c3-5ad2ee968720 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.085082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e08f7-6b24-456c-81ff-a28795ca70b9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-257e08f7-6b24-456c-81ff-a28795ca70b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:55.098808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:55.100054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 11:36:55.101253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:55.103585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 344/659] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:55.104800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e08f7-6b24-456c-81ff-a28795ca70b9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.173989 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:55.196142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a91412-e454-44c1-b6c3-5ad2ee968720 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.209388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e08f7-6b24-456c-81ff-a28795ca70b9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.234009 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893b85-aab2-427b-a07b-2ee415830e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.234724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893b85-aab2-427b-a07b-2ee415830e5a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.234998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893b85-aab2-427b-a07b-2ee415830e5a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.235318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893b85-aab2-427b-a07b-2ee415830e5a None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.236034 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893b85-aab2-427b-a07b-2ee415830e5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:55.236370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893b85-aab2-427b-a07b-2ee415830e5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.237809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893b85-aab2-427b-a07b-2ee415830e5a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.243005 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b297976-e918-4edc-a2c4-f2f0ea909555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.243662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b297976-e918-4edc-a2c4-f2f0ea909555 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.243935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b297976-e918-4edc-a2c4-f2f0ea909555 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.244198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b297976-e918-4edc-a2c4-f2f0ea909555 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.251327 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b297976-e918-4edc-a2c4-f2f0ea909555 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.257163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b297976-e918-4edc-a2c4-f2f0ea909555 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.257294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 316/660] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:55.257294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893b85-aab2-427b-a07b-2ee415830e5a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-6b297976-e918-4edc-a2c4-f2f0ea909555 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:55.268604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:55.269656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:55.270649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:55.270649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:55.270649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:55.270649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:55.270649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:55.270649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:55.270649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:55.270649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:55.270649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 345/661] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:55.271160 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b297976-e918-4edc-a2c4-f2f0ea909555 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.312823 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:55.359020 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893b85-aab2-427b-a07b-2ee415830e5a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.377257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b297976-e918-4edc-a2c4-f2f0ea909555 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.391300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870034b-f06a-40d1-baec-bb3dc95a1263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.391905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870034b-f06a-40d1-baec-bb3dc95a1263 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.392183 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870034b-f06a-40d1-baec-bb3dc95a1263 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.392445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870034b-f06a-40d1-baec-bb3dc95a1263 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.393166 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870034b-f06a-40d1-baec-bb3dc95a1263 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:55.393471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870034b-f06a-40d1-baec-bb3dc95a1263 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.394893 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870034b-f06a-40d1-baec-bb3dc95a1263 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.402506 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:55.414468 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 317/662] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:55.415406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870034b-f06a-40d1-baec-bb3dc95a1263 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.418622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f5f5a-b648-45e1-b2a2-4c3c73c5d5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.419402 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f5f5a-b648-45e1-b2a2-4c3c73c5d5ee None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.419797 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f5f5a-b648-45e1-b2a2-4c3c73c5d5ee None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.420165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f5f5a-b648-45e1-b2a2-4c3c73c5d5ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.432458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f5f5a-b648-45e1-b2a2-4c3c73c5d5ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-046f5f5a-b648-45e1-b2a2-4c3c73c5d5ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:55.438883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:55.440071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:36:55.441662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:55.444012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 346/663] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:55.444907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f5f5a-b648-45e1-b2a2-4c3c73c5d5ee None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.534001 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870034b-f06a-40d1-baec-bb3dc95a1263 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.544248 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:55.548028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f5f5a-b648-45e1-b2a2-4c3c73c5d5ee None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.569499 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470832ae-db7a-467d-ad9f-dc89f9774dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.570123 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470832ae-db7a-467d-ad9f-dc89f9774dba None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.570398 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470832ae-db7a-467d-ad9f-dc89f9774dba None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.570666 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470832ae-db7a-467d-ad9f-dc89f9774dba None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.577868 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470832ae-db7a-467d-ad9f-dc89f9774dba None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.579609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470832ae-db7a-467d-ad9f-dc89f9774dba None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.585887 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f827bb-271b-44a2-85e1-14c547273de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.586494 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f827bb-271b-44a2-85e1-14c547273de7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.586766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f827bb-271b-44a2-85e1-14c547273de7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.587032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f827bb-271b-44a2-85e1-14c547273de7 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.587755 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f827bb-271b-44a2-85e1-14c547273de7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:55.588057 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f827bb-271b-44a2-85e1-14c547273de7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.589469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f827bb-271b-44a2-85e1-14c547273de7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-470832ae-db7a-467d-ad9f-dc89f9774dba None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:55.591218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:55.592118 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:55.593054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 318/664] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:55.593654 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470832ae-db7a-467d-ad9f-dc89f9774dba None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.611500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 347/665] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:55.611500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f827bb-271b-44a2-85e1-14c547273de7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.657576 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:55.704406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470832ae-db7a-467d-ad9f-dc89f9774dba None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.711155 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f827bb-271b-44a2-85e1-14c547273de7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.738428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a630d8-62a2-44d8-b2d0-8bec71b7e0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.739108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a630d8-62a2-44d8-b2d0-8bec71b7e0f4 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.739406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a630d8-62a2-44d8-b2d0-8bec71b7e0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.739677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a630d8-62a2-44d8-b2d0-8bec71b7e0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.748325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54580d7-fede-4e3c-b601-4f402f07219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.749056 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54580d7-fede-4e3c-b601-4f402f07219c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.749334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54580d7-fede-4e3c-b601-4f402f07219c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.749600 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54580d7-fede-4e3c-b601-4f402f07219c None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.750286 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54580d7-fede-4e3c-b601-4f402f07219c None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:55.750586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54580d7-fede-4e3c-b601-4f402f07219c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.752057 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54580d7-fede-4e3c-b601-4f402f07219c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.767167 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 348/666] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:55.767167 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54580d7-fede-4e3c-b601-4f402f07219c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.794683 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:55.810267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a630d8-62a2-44d8-b2d0-8bec71b7e0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.811785 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a630d8-62a2-44d8-b2d0-8bec71b7e0f4 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.865042 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:55.879810 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54580d7-fede-4e3c-b601-4f402f07219c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:55.913370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 319/667] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:55.913370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a630d8-62a2-44d8-b2d0-8bec71b7e0f4 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:55.918928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bcd26-7944-4e1f-99b3-4790ba9c8e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:55.919663 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bcd26-7944-4e1f-99b3-4790ba9c8e0c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:55.919941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bcd26-7944-4e1f-99b3-4790ba9c8e0c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:55.920273 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bcd26-7944-4e1f-99b3-4790ba9c8e0c None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:55.921153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bcd26-7944-4e1f-99b3-4790ba9c8e0c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:55.921498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bcd26-7944-4e1f-99b3-4790ba9c8e0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:55.922952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bcd26-7944-4e1f-99b3-4790ba9c8e0c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:55.946084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 349/668] 10.222.0.16 () {56 vars in 1139 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:55.946084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bcd26-7944-4e1f-99b3-4790ba9c8e0c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.013804 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a630d8-62a2-44d8-b2d0-8bec71b7e0f4 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.021246 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:56.043317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bcd26-7944-4e1f-99b3-4790ba9c8e0c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.050028 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea45eae-e598-447d-b818-8128c0581a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.051160 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea45eae-e598-447d-b818-8128c0581a19 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.051608 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea45eae-e598-447d-b818-8128c0581a19 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.052041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea45eae-e598-447d-b818-8128c0581a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.052906 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea45eae-e598-447d-b818-8128c0581a19 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=efbe019e4d4646428bed474cef4009b8) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:56.053352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea45eae-e598-447d-b818-8128c0581a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.054936 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea45eae-e598-447d-b818-8128c0581a19 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:56.061036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 320/669] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/role_assignments?user.id=efbe019e4d4646428bed474cef4009b8&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:56.061884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea45eae-e598-447d-b818-8128c0581a19 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.088125 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d586eb-403b-425a-9836-8f7ae42fb50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.089371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d586eb-403b-425a-9836-8f7ae42fb50a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.089787 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d586eb-403b-425a-9836-8f7ae42fb50a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.090203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d586eb-403b-425a-9836-8f7ae42fb50a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.098988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:56.101948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d586eb-403b-425a-9836-8f7ae42fb50a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.103512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d586eb-403b-425a-9836-8f7ae42fb50a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-78d586eb-403b-425a-9836-8f7ae42fb50a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:56.119493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:56.120500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:56.121367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:56.121367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:56.121367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:56.125649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 350/670] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:56.126652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d586eb-403b-425a-9836-8f7ae42fb50a None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.218068 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea45eae-e598-447d-b818-8128c0581a19 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.244346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d586eb-403b-425a-9836-8f7ae42fb50a None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.256950 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58252ff7-145d-4760-b811-13baeda65cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.257595 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58252ff7-145d-4760-b811-13baeda65cef None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.257869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58252ff7-145d-4760-b811-13baeda65cef None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.258233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58252ff7-145d-4760-b811-13baeda65cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.271496 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58252ff7-145d-4760-b811-13baeda65cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.273226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58252ff7-145d-4760-b811-13baeda65cef None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:56.276394 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:56.286213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96337543-0171-461a-8fb5-0f8000603eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.287023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96337543-0171-461a-8fb5-0f8000603eb3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.287418 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96337543-0171-461a-8fb5-0f8000603eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.287780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96337543-0171-461a-8fb5-0f8000603eb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-58252ff7-145d-4760-b811-13baeda65cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:56.291407 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:56.292430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:56.293314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:56.293314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:56.293314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:56.293314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 321/671] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:56.293932 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58252ff7-145d-4760-b811-13baeda65cef None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.369275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96337543-0171-461a-8fb5-0f8000603eb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.370667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96337543-0171-461a-8fb5-0f8000603eb3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:56.379231 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:56.422701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58252ff7-145d-4760-b811-13baeda65cef None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.477475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a0e6f-bc16-41b7-bf00-cf903eb5487b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.478089 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a0e6f-bc16-41b7-bf00-cf903eb5487b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.478363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a0e6f-bc16-41b7-bf00-cf903eb5487b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.478631 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a0e6f-bc16-41b7-bf00-cf903eb5487b None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.479318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a0e6f-bc16-41b7-bf00-cf903eb5487b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:56.479623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a0e6f-bc16-41b7-bf00-cf903eb5487b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.481026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a0e6f-bc16-41b7-bf00-cf903eb5487b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:56.483725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 351/672] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:56.484854 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96337543-0171-461a-8fb5-0f8000603eb3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.497828 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 322/673] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:56.498625 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a0e6f-bc16-41b7-bf00-cf903eb5487b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.517559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:56.592386 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96337543-0171-461a-8fb5-0f8000603eb3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.596453 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a0e6f-bc16-41b7-bf00-cf903eb5487b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.630323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d792b3-853d-4989-b82e-24adebb29383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.630913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d792b3-853d-4989-b82e-24adebb29383 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.631219 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d792b3-853d-4989-b82e-24adebb29383 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.631819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d792b3-853d-4989-b82e-24adebb29383 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.632251 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d792b3-853d-4989-b82e-24adebb29383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:56.632477 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:56.632581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d792b3-853d-4989-b82e-24adebb29383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.633920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d792b3-853d-4989-b82e-24adebb29383 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:56.637950 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36d550-74fa-47f8-8ae9-6893a961e062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.638865 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36d550-74fa-47f8-8ae9-6893a961e062 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.639513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36d550-74fa-47f8-8ae9-6893a961e062 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.639778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36d550-74fa-47f8-8ae9-6893a961e062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.647457 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 352/674] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:56.648104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d792b3-853d-4989-b82e-24adebb29383 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.655953 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36d550-74fa-47f8-8ae9-6893a961e062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.658197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36d550-74fa-47f8-8ae9-6893a961e062 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-2e36d550-74fa-47f8-8ae9-6893a961e062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:56.676332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:56.677296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:56.678194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:56.678194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:56.678194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:56.678194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 323/675] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:56.679312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b00844-b5cd-4ac2-90b0-158da401bcd5 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.679312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b00844-b5cd-4ac2-90b0-158da401bcd5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.679312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b00844-b5cd-4ac2-90b0-158da401bcd5 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.683239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 324/676] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:56.684529 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b00844-b5cd-4ac2-90b0-158da401bcd5 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.752611 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:56.763146 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d792b3-853d-4989-b82e-24adebb29383 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.797561 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b00844-b5cd-4ac2-90b0-158da401bcd5 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.799191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf5928-1346-48b2-8815-ab5ca57cef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.799838 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf5928-1346-48b2-8815-ab5ca57cef40 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.800121 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf5928-1346-48b2-8815-ab5ca57cef40 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.800385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf5928-1346-48b2-8815-ab5ca57cef40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.808446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf5928-1346-48b2-8815-ab5ca57cef40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.809949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf5928-1346-48b2-8815-ab5ca57cef40 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-cfaf5928-1346-48b2-8815-ab5ca57cef40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:56.821413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:56.822410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 353/677] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:36:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:56.823515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf5928-1346-48b2-8815-ab5ca57cef40 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.841334 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f74416-87b0-47ff-8d01-c499b2aaba27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.842302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f74416-87b0-47ff-8d01-c499b2aaba27 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.842735 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f74416-87b0-47ff-8d01-c499b2aaba27 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.843187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f74416-87b0-47ff-8d01-c499b2aaba27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.850622 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f74416-87b0-47ff-8d01-c499b2aaba27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.852378 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f74416-87b0-47ff-8d01-c499b2aaba27 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-04f74416-87b0-47ff-8d01-c499b2aaba27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:56.864650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:56.865724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:56.866773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:56.866773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:56.866773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:56.866773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:56.866773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:56.866773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:56.866773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:56.866773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:56.870563 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 325/678] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:56.871837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9383f-86bd-4ab9-9263-06ddc413237e None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.872119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9383f-86bd-4ab9-9263-06ddc413237e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.872390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9383f-86bd-4ab9-9263-06ddc413237e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.891543 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-97b9383f-86bd-4ab9-9263-06ddc413237e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:56.892759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-97b9383f-86bd-4ab9-9263-06ddc413237e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:56.926955 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf5928-1346-48b2-8815-ab5ca57cef40 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:56.934843 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:56.963931 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 326/679] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:36:56.964987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9383f-86bd-4ab9-9263-06ddc413237e None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:56.981408 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6196b9-ea9d-4197-a79b-93c3776cb616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:56.982037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6196b9-ea9d-4197-a79b-93c3776cb616 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:56.982327 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6196b9-ea9d-4197-a79b-93c3776cb616 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:56.982574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6196b9-ea9d-4197-a79b-93c3776cb616 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:56.983314 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6196b9-ea9d-4197-a79b-93c3776cb616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:56.983526 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6196b9-ea9d-4197-a79b-93c3776cb616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:56.984925 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6196b9-ea9d-4197-a79b-93c3776cb616 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:57.006406 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 354/680] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:57.007359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6196b9-ea9d-4197-a79b-93c3776cb616 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.020249 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:57.081079 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9383f-86bd-4ab9-9263-06ddc413237e None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:57.121113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba83db-52b8-4f5d-85e8-24973ad9d068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:57.121756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba83db-52b8-4f5d-85e8-24973ad9d068 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:57.122030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba83db-52b8-4f5d-85e8-24973ad9d068 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:57.122353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba83db-52b8-4f5d-85e8-24973ad9d068 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:57.123211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba83db-52b8-4f5d-85e8-24973ad9d068 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:57.123726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba83db-52b8-4f5d-85e8-24973ad9d068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:57.125495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba83db-52b8-4f5d-85e8-24973ad9d068 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:57.133788 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6196b9-ea9d-4197-a79b-93c3776cb616 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:57.150098 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 327/681] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Aug 30 11:36:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:57.151362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba83db-52b8-4f5d-85e8-24973ad9d068 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.174775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216a7e-227b-435f-b51c-91f8c346e6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:57.175452 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216a7e-227b-435f-b51c-91f8c346e6f3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:57.175727 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216a7e-227b-435f-b51c-91f8c346e6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:57.175994 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216a7e-227b-435f-b51c-91f8c346e6f3 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:57.176790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216a7e-227b-435f-b51c-91f8c346e6f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:57.177090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216a7e-227b-435f-b51c-91f8c346e6f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:57.178538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216a7e-227b-435f-b51c-91f8c346e6f3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:57.190982 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:57.191215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 355/682] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:57.192088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216a7e-227b-435f-b51c-91f8c346e6f3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.270822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba83db-52b8-4f5d-85e8-24973ad9d068 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:57.274349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:57.307269 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef0fd6-68e5-41c3-8636-efa426a36bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:57.307904 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef0fd6-68e5-41c3-8636-efa426a36bf3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:57.308180 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef0fd6-68e5-41c3-8636-efa426a36bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:57.308450 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef0fd6-68e5-41c3-8636-efa426a36bf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:57.310777 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216a7e-227b-435f-b51c-91f8c346e6f3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:57.315229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef0fd6-68e5-41c3-8636-efa426a36bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:57.316704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef0fd6-68e5-41c3-8636-efa426a36bf3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-58ef0fd6-68e5-41c3-8636-efa426a36bf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:57.329405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:57.330391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:57.331353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:57.331353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:57.331353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:57.331353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:57.331353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:57.331353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:57.331353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:57.331353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:57.331353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 328/683] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:57.332684 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef0fd6-68e5-41c3-8636-efa426a36bf3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.354109 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da7ae1-8332-4f33-b24d-397f5c7cfe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:57.355063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da7ae1-8332-4f33-b24d-397f5c7cfe53 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:57.355576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da7ae1-8332-4f33-b24d-397f5c7cfe53 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:57.356050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da7ae1-8332-4f33-b24d-397f5c7cfe53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:57.432879 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:57.434834 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da7ae1-8332-4f33-b24d-397f5c7cfe53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:57.438832 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da7ae1-8332-4f33-b24d-397f5c7cfe53 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:57.445502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef0fd6-68e5-41c3-8636-efa426a36bf3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:57.481893 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d111f1c-c431-47ea-a724-42ad7e210799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:57.482491 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d111f1c-c431-47ea-a724-42ad7e210799 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:57.482765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d111f1c-c431-47ea-a724-42ad7e210799 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:57.483031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d111f1c-c431-47ea-a724-42ad7e210799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:57.483890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d111f1c-c431-47ea-a724-42ad7e210799 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=d3250e8f23c64f26b4e862090f7f2e18) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:57.484325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d111f1c-c431-47ea-a724-42ad7e210799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:57.485754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d111f1c-c431-47ea-a724-42ad7e210799 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:57.499631 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 329/684] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/role_assignments?user.id=d3250e8f23c64f26b4e862090f7f2e18&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:57.501041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d111f1c-c431-47ea-a724-42ad7e210799 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.505276 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:57.546106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 356/685] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:57.547505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da7ae1-8332-4f33-b24d-397f5c7cfe53 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.621102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d111f1c-c431-47ea-a724-42ad7e210799 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:57.656162 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da7ae1-8332-4f33-b24d-397f5c7cfe53 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:57.660237 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108effd-fcb8-41c0-8870-119851ddd7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:57.661001 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108effd-fcb8-41c0-8870-119851ddd7d5 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:57.661277 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108effd-fcb8-41c0-8870-119851ddd7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:57.661541 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108effd-fcb8-41c0-8870-119851ddd7d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:57.673364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108effd-fcb8-41c0-8870-119851ddd7d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-2108effd-fcb8-41c0-8870-119851ddd7d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:36:57.680257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:57.681196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:36:57.682046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:57.688390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 330/686] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:57.688520 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108effd-fcb8-41c0-8870-119851ddd7d5 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.703411 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3978d6c-8d7d-4eca-8d51-ad9961b2d9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:57.704346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3978d6c-8d7d-4eca-8d51-ad9961b2d9c2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:57.704624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3978d6c-8d7d-4eca-8d51-ad9961b2d9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:57.704885 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3978d6c-8d7d-4eca-8d51-ad9961b2d9c2 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:57.705575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3978d6c-8d7d-4eca-8d51-ad9961b2d9c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:57.705875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3978d6c-8d7d-4eca-8d51-ad9961b2d9c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:57.707400 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3978d6c-8d7d-4eca-8d51-ad9961b2d9c2 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:57.717112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:57.720596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 357/687] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:57.721465 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3978d6c-8d7d-4eca-8d51-ad9961b2d9c2 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.763518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:57.819168 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108effd-fcb8-41c0-8870-119851ddd7d5 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:57.857643 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3978d6c-8d7d-4eca-8d51-ad9961b2d9c2 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:57.861339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d0780-041b-471c-bb50-356c0efd3455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:57.861599 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d0780-041b-471c-bb50-356c0efd3455 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:57.861968 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d0780-041b-471c-bb50-356c0efd3455 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:57.862361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d0780-041b-471c-bb50-356c0efd3455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:57.873847 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d0780-041b-471c-bb50-356c0efd3455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:57.875464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d0780-041b-471c-bb50-356c0efd3455 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-b22d0780-041b-471c-bb50-356c0efd3455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:57.890272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:57.891564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:57.892720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:57.892720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:57.892720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:57.894266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 331/688] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:57.895248 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d0780-041b-471c-bb50-356c0efd3455 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.903652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827584b-41de-4faa-a577-f36c08cdb6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:57.904529 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827584b-41de-4faa-a577-f36c08cdb6fe None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:57.904963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827584b-41de-4faa-a577-f36c08cdb6fe None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:57.905939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827584b-41de-4faa-a577-f36c08cdb6fe None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:57.906786 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827584b-41de-4faa-a577-f36c08cdb6fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:57.907302 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827584b-41de-4faa-a577-f36c08cdb6fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:57.909400 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827584b-41de-4faa-a577-f36c08cdb6fe None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:57.928531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 358/689] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:57.928531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827584b-41de-4faa-a577-f36c08cdb6fe None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:57.974233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:58.011712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d0780-041b-471c-bb50-356c0efd3455 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:58.048749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3940076-32d8-486a-9b12-cb2db2888301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:58.049360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3940076-32d8-486a-9b12-cb2db2888301 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.049633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3940076-32d8-486a-9b12-cb2db2888301 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.049900 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3940076-32d8-486a-9b12-cb2db2888301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.050906 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3940076-32d8-486a-9b12-cb2db2888301 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=56bc4fdd14d147ef86b96786da2855f5) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:58.051033 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3940076-32d8-486a-9b12-cb2db2888301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:58.052566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3940076-32d8-486a-9b12-cb2db2888301 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:58.054972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:58.064957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827584b-41de-4faa-a577-f36c08cdb6fe None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:58.065580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 332/690] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/role_assignments?user.id=56bc4fdd14d147ef86b96786da2855f5&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:58.066720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3940076-32d8-486a-9b12-cb2db2888301 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:58.108178 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c0345-b31a-478b-b908-99a042cac816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:58.108178 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c0345-b31a-478b-b908-99a042cac816 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.108178 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c0345-b31a-478b-b908-99a042cac816 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.108178 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c0345-b31a-478b-b908-99a042cac816 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.108621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c0345-b31a-478b-b908-99a042cac816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:58.108621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c0345-b31a-478b-b908-99a042cac816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:58.108621 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c0345-b31a-478b-b908-99a042cac816 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:58.124848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 359/691] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:58.126277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c0345-b31a-478b-b908-99a042cac816 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:58.213245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3940076-32d8-486a-9b12-cb2db2888301 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:58.244475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:58.250172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465d494-26f2-4da2-a5f3-c376cac8bb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:58.250898 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465d494-26f2-4da2-a5f3-c376cac8bb42 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.251079 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465d494-26f2-4da2-a5f3-c376cac8bb42 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.251391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465d494-26f2-4da2-a5f3-c376cac8bb42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.257577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c0345-b31a-478b-b908-99a042cac816 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:58.262811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465d494-26f2-4da2-a5f3-c376cac8bb42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:58.264302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465d494-26f2-4da2-a5f3-c376cac8bb42 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-c465d494-26f2-4da2-a5f3-c376cac8bb42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:58.283240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:58.284232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:58.285138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:58.285138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:58.285138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:58.285138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 333/692] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:58.286047 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba7793-e752-43f2-a3d2-ae846488985a None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.286301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba7793-e752-43f2-a3d2-ae846488985a None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.286571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba7793-e752-43f2-a3d2-ae846488985a None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.288089 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 334/693] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:58.293637 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de0417-cb62-4f19-a14b-e04564d952b0 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.294221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de0417-cb62-4f19-a14b-e04564d952b0 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.294592 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de0417-cb62-4f19-a14b-e04564d952b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.308100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3095e-72bd-49ff-ba8e-ae544795c0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:58.308977 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3095e-72bd-49ff-ba8e-ae544795c0cb None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.309427 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3095e-72bd-49ff-ba8e-ae544795c0cb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.309867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3095e-72bd-49ff-ba8e-ae544795c0cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.316635 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-39de0417-cb62-4f19-a14b-e04564d952b0 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:58.318003 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-39de0417-cb62-4f19-a14b-e04564d952b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:58.318135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3095e-72bd-49ff-ba8e-ae544795c0cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:58.319883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3095e-72bd-49ff-ba8e-ae544795c0cb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-a8c3095e-72bd-49ff-ba8e-ae544795c0cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:36:58.335924 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:58.337106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:58.338250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:36:58.338250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:36:58.338250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:36:58.338250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:36:58.338250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:36:58.338250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:36:58.338250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:36:58.338250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:58.345020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 360/694] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:58.345020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3095e-72bd-49ff-ba8e-ae544795c0cb None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:58.357934 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:58.405109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 335/695] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:58.406105 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de0417-cb62-4f19-a14b-e04564d952b0 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:58.485692 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3095e-72bd-49ff-ba8e-ae544795c0cb None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:58.556627 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97072d6d-433a-4196-b3dd-8e5f3ee14f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:58.557175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97072d6d-433a-4196-b3dd-8e5f3ee14f04 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.557448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97072d6d-433a-4196-b3dd-8e5f3ee14f04 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.557714 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97072d6d-433a-4196-b3dd-8e5f3ee14f04 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.558473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97072d6d-433a-4196-b3dd-8e5f3ee14f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:58.558827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97072d6d-433a-4196-b3dd-8e5f3ee14f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:58.560410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97072d6d-433a-4196-b3dd-8e5f3ee14f04 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:58.583460 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 361/696] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:58.584378 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de0417-cb62-4f19-a14b-e04564d952b0 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:58.588049 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97072d6d-433a-4196-b3dd-8e5f3ee14f04 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:58.610623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:58.648957 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f10ef-6b7e-4e57-a3f8-64e57089188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:58.650358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f10ef-6b7e-4e57-a3f8-64e57089188d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.650635 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f10ef-6b7e-4e57-a3f8-64e57089188d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.650901 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f10ef-6b7e-4e57-a3f8-64e57089188d None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.651636 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f10ef-6b7e-4e57-a3f8-64e57089188d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:58.651938 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f10ef-6b7e-4e57-a3f8-64e57089188d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:58.653386 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f10ef-6b7e-4e57-a3f8-64e57089188d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:58.676984 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 336/697] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:36:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:58.682657 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f10ef-6b7e-4e57-a3f8-64e57089188d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:58.716975 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:58.748175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97072d6d-433a-4196-b3dd-8e5f3ee14f04 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:58.782922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291b2a1-f0a8-493f-ab22-12b5d7c35c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:58.783850 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291b2a1-f0a8-493f-ab22-12b5d7c35c8a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.784268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291b2a1-f0a8-493f-ab22-12b5d7c35c8a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.784696 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291b2a1-f0a8-493f-ab22-12b5d7c35c8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.792305 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291b2a1-f0a8-493f-ab22-12b5d7c35c8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:58.793950 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291b2a1-f0a8-493f-ab22-12b5d7c35c8a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-2291b2a1-f0a8-493f-ab22-12b5d7c35c8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:36:58.805942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:58.806947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:36:58.807948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:58.810194 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 362/698] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Aug 30 11:36:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:58.811273 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291b2a1-f0a8-493f-ab22-12b5d7c35c8a None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:58.811557 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f10ef-6b7e-4e57-a3f8-64e57089188d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:58.855239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d534fb-c733-4f72-b761-9f86d0c1263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:58.856365 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d534fb-c733-4f72-b761-9f86d0c1263c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.856708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d534fb-c733-4f72-b761-9f86d0c1263c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.857040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d534fb-c733-4f72-b761-9f86d0c1263c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.871098 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d534fb-c733-4f72-b761-9f86d0c1263c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:58.873324 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d534fb-c733-4f72-b761-9f86d0c1263c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-19d534fb-c733-4f72-b761-9f86d0c1263c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:58.892176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:58.893223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:58.894151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:58.894151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:58.894151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:36:58.895449 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 337/699] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:36:58.896511 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d534fb-c733-4f72-b761-9f86d0c1263c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:58.921352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291b2a1-f0a8-493f-ab22-12b5d7c35c8a None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:58.933406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:58.952053 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:58.958596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7b18d-d778-46c0-aab9-e6f02c4a0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:58.959253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7b18d-d778-46c0-aab9-e6f02c4a0cd2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:58.959524 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7b18d-d778-46c0-aab9-e6f02c4a0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:58.959846 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7b18d-d778-46c0-aab9-e6f02c4a0cd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:58.960575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7b18d-d778-46c0-aab9-e6f02c4a0cd2 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=8bbb8fb036684305bc00290c519939ec) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:58.960877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7b18d-d778-46c0-aab9-e6f02c4a0cd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:58.962311 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7b18d-d778-46c0-aab9-e6f02c4a0cd2 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:58.968665 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 363/700] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:36:58 2022] GET /identity/v3/role_assignments?user.id=8bbb8fb036684305bc00290c519939ec&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:58.969544 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7b18d-d778-46c0-aab9-e6f02c4a0cd2 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:59.028243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d534fb-c733-4f72-b761-9f86d0c1263c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:59.065285 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef7f73-b0ed-4378-a657-85db9a5074fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:59.066264 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef7f73-b0ed-4378-a657-85db9a5074fe None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:59.066769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef7f73-b0ed-4378-a657-85db9a5074fe None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:59.067374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef7f73-b0ed-4378-a657-85db9a5074fe None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:59.068317 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef7f73-b0ed-4378-a657-85db9a5074fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:59.068847 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef7f73-b0ed-4378-a657-85db9a5074fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:59.070548 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef7f73-b0ed-4378-a657-85db9a5074fe None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:59.091890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 338/701] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Aug 30 11:36:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:59.097940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef7f73-b0ed-4378-a657-85db9a5074fe None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:59.099533 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7b18d-d778-46c0-aab9-e6f02c4a0cd2 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:59.178494 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65d53d-eca9-4731-95f4-928d31328a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:59.179517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65d53d-eca9-4731-95f4-928d31328a7f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:59.180569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65d53d-eca9-4731-95f4-928d31328a7f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:59.180937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65d53d-eca9-4731-95f4-928d31328a7f None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:59.181976 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65d53d-eca9-4731-95f4-928d31328a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:59.189757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65d53d-eca9-4731-95f4-928d31328a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:59.189757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65d53d-eca9-4731-95f4-928d31328a7f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:59.204742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 364/702] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:59.212289 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65d53d-eca9-4731-95f4-928d31328a7f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:59.241998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:59.260409 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef7f73-b0ed-4378-a657-85db9a5074fe None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:59.283603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:59.297708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b56ab-bcb5-4f67-ae31-93a78992ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:59.298319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b56ab-bcb5-4f67-ae31-93a78992ec86 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:59.298592 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b56ab-bcb5-4f67-ae31-93a78992ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:59.298854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b56ab-bcb5-4f67-ae31-93a78992ec86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:59.330100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65d53d-eca9-4731-95f4-928d31328a7f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:59.367313 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee68c01-3248-47f7-adf4-e0a98d0c32c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:59.368163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee68c01-3248-47f7-adf4-e0a98d0c32c1 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:59.368534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee68c01-3248-47f7-adf4-e0a98d0c32c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:59.368971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee68c01-3248-47f7-adf4-e0a98d0c32c1 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:59.369728 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee68c01-3248-47f7-adf4-e0a98d0c32c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:59.371155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b56ab-bcb5-4f67-ae31-93a78992ec86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:59.371386 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee68c01-3248-47f7-adf4-e0a98d0c32c1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:59.372706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b56ab-bcb5-4f67-ae31-93a78992ec86 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:59.401101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 365/703] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:36:59 2022] POST /identity/v3/users => generated 254 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:36:59.476593 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:59.498674 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 339/704] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:36:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:36:59.504023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee68c01-3248-47f7-adf4-e0a98d0c32c1 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:59.566936 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79683bd0-b9d0-4359-b1d1-82cdc02e20bb None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:59.567229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79683bd0-b9d0-4359-b1d1-82cdc02e20bb None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:59.567505 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79683bd0-b9d0-4359-b1d1-82cdc02e20bb None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:59.569005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 340/705] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:36:59.575014 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234e53c-157e-4a9b-bd96-c02cd8c0cbd7 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:59.575335 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234e53c-157e-4a9b-bd96-c02cd8c0cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:59.575609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234e53c-157e-4a9b-bd96-c02cd8c0cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:59.603880 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-5234e53c-157e-4a9b-bd96-c02cd8c0cbd7 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:36:59.606098 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-5234e53c-157e-4a9b-bd96-c02cd8c0cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:36:59.624135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee68c01-3248-47f7-adf4-e0a98d0c32c1 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:59.641381 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:59.671224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be9bdb-e6ea-4882-b307-efb84a326a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:59.672220 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9bdb-e6ea-4882-b307-efb84a326a09 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:59.672894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9bdb-e6ea-4882-b307-efb84a326a09 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:59.673805 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9bdb-e6ea-4882-b307-efb84a326a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:59.688426 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be9bdb-e6ea-4882-b307-efb84a326a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:59.690997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be9bdb-e6ea-4882-b307-efb84a326a09 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:59.694905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 341/706] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-03be9bdb-e6ea-4882-b307-efb84a326a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:36:59.710922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:36:59.711939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:36:59.712824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:36:59.712824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:36:59.712824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:36:59.712824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 366/707] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:36:59.719089 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234e53c-157e-4a9b-bd96-c02cd8c0cbd7 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:36:59.836611 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234e53c-157e-4a9b-bd96-c02cd8c0cbd7 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:36:59.852188 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:36:59.879157 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec6d7e-d25d-4606-b155-e23dbce55624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:36:59.879768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec6d7e-d25d-4606-b155-e23dbce55624 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:36:59.880042 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec6d7e-d25d-4606-b155-e23dbce55624 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:36:59.880303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec6d7e-d25d-4606-b155-e23dbce55624 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:36:59.880958 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec6d7e-d25d-4606-b155-e23dbce55624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:36:59.881259 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec6d7e-d25d-4606-b155-e23dbce55624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:36:59.882644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec6d7e-d25d-4606-b155-e23dbce55624 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:36:59.900465 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 342/708] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:36:59.906203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be9bdb-e6ea-4882-b307-efb84a326a09 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:00.001679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec6d7e-d25d-4606-b155-e23dbce55624 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:00.018465 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be9bdb-e6ea-4882-b307-efb84a326a09 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:00.059158 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85237691-ec89-4ff5-b8f7-d18bb8b7ffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:00.060166 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85237691-ec89-4ff5-b8f7-d18bb8b7ffb7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:00.060815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85237691-ec89-4ff5-b8f7-d18bb8b7ffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:00.061279 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85237691-ec89-4ff5-b8f7-d18bb8b7ffb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:00.069131 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85237691-ec89-4ff5-b8f7-d18bb8b7ffb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:00.071047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85237691-ec89-4ff5-b8f7-d18bb8b7ffb7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-85237691-ec89-4ff5-b8f7-d18bb8b7ffb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:00.085790 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:00.086735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:00.087653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:00.087653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:00.087653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:00.087653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:00.087653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:00.087653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:00.087653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:00.087653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:00.090234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 367/709] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:36:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:00.096339 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85237691-ec89-4ff5-b8f7-d18bb8b7ffb7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:00.125456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:00.193246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec6d7e-d25d-4606-b155-e23dbce55624 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:00.210350 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85237691-ec89-4ff5-b8f7-d18bb8b7ffb7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:00.252021 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493114d-56fd-4e05-9036-4842fb95b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:00.252990 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493114d-56fd-4e05-9036-4842fb95b939 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:00.253473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493114d-56fd-4e05-9036-4842fb95b939 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:00.255323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493114d-56fd-4e05-9036-4842fb95b939 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:00.256819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493114d-56fd-4e05-9036-4842fb95b939 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:00.257316 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493114d-56fd-4e05-9036-4842fb95b939 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:00.260127 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493114d-56fd-4e05-9036-4842fb95b939 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:00.275191 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ce304-a885-43ec-adc4-e5c817369f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:00.276586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ce304-a885-43ec-adc4-e5c817369f24 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:00.277071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ce304-a885-43ec-adc4-e5c817369f24 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:00.277528 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ce304-a885-43ec-adc4-e5c817369f24 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:00.278319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ce304-a885-43ec-adc4-e5c817369f24 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:00.280597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ce304-a885-43ec-adc4-e5c817369f24 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:00.289292 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 368/710] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-584ce304-a885-43ec-adc4-e5c817369f24 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:00.298609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:00.299687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 11:37:00.300612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:00.303475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 343/711] 10.222.0.16 () {56 vars in 1129 bytes} [Tue Aug 30 11:37:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:00.311603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493114d-56fd-4e05-9036-4842fb95b939 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:00.314075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ce304-a885-43ec-adc4-e5c817369f24 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:00.397550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:00.427017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:00.450670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493114d-56fd-4e05-9036-4842fb95b939 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:00.489079 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c8e4a-0f38-4139-bc7e-418f7e0e2fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:00.489604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c8e4a-0f38-4139-bc7e-418f7e0e2fcc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:00.489875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c8e4a-0f38-4139-bc7e-418f7e0e2fcc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:00.490138 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c8e4a-0f38-4139-bc7e-418f7e0e2fcc None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:00.490762 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c8e4a-0f38-4139-bc7e-418f7e0e2fcc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:00.491086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c8e4a-0f38-4139-bc7e-418f7e0e2fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:00.492444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c8e4a-0f38-4139-bc7e-418f7e0e2fcc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:00.493296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ce304-a885-43ec-adc4-e5c817369f24 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:00.497498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 369/712] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Aug 30 11:37:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:00.503858 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc7eeb-71e1-47ef-8bbb-4acf34cc7dbd None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:00.504021 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc7eeb-71e1-47ef-8bbb-4acf34cc7dbd None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:00.504399 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc7eeb-71e1-47ef-8bbb-4acf34cc7dbd None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:00.509927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 370/713] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:00.509927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc7eeb-71e1-47ef-8bbb-4acf34cc7dbd None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:00.539371 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b5354-ce6b-4337-9a18-8d6508fb227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:00.540339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b5354-ce6b-4337-9a18-8d6508fb227e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:00.540819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b5354-ce6b-4337-9a18-8d6508fb227e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:00.541283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b5354-ce6b-4337-9a18-8d6508fb227e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:00.542226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b5354-ce6b-4337-9a18-8d6508fb227e None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=0bb80229c3234d5f9599b16870c69934) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:00.542727 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b5354-ce6b-4337-9a18-8d6508fb227e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:00.544403 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b5354-ce6b-4337-9a18-8d6508fb227e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:00.551031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 344/714] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:00 2022] GET /identity/v3/role_assignments?user.id=0bb80229c3234d5f9599b16870c69934&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:00.552572 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68873-a1bc-4bff-b628-a55d1c6b8c22 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:00.553048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68873-a1bc-4bff-b628-a55d1c6b8c22 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:00.553510 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68873-a1bc-4bff-b628-a55d1c6b8c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:00.572962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-cfb68873-a1bc-4bff-b628-a55d1c6b8c22 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:00.574501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-cfb68873-a1bc-4bff-b628-a55d1c6b8c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:00.690763 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 345/715] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:37:00.691906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc7eeb-71e1-47ef-8bbb-4acf34cc7dbd None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:00.744116 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c682f48-0d88-40cf-a205-5ca4f5e584f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:00.745161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c682f48-0d88-40cf-a205-5ca4f5e584f7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:00.745635 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c682f48-0d88-40cf-a205-5ca4f5e584f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:00.746097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c682f48-0d88-40cf-a205-5ca4f5e584f7 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:00.746950 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c682f48-0d88-40cf-a205-5ca4f5e584f7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:00.747464 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c682f48-0d88-40cf-a205-5ca4f5e584f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:00.749241 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c682f48-0d88-40cf-a205-5ca4f5e584f7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:00.758954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 371/716] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Aug 30 11:37:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:00.994762 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb68873-a1bc-4bff-b628-a55d1c6b8c22 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:01.029954 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:01.101463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb68873-a1bc-4bff-b628-a55d1c6b8c22 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:01.140559 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05f599-c5c5-4156-9e27-a1befa67cae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:01.141632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05f599-c5c5-4156-9e27-a1befa67cae3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:01.142148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05f599-c5c5-4156-9e27-a1befa67cae3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:01.142649 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05f599-c5c5-4156-9e27-a1befa67cae3 None admin] PATH_INFO: `/v3/services/network` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:01.143464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05f599-c5c5-4156-9e27-a1befa67cae3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:01.145094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05f599-c5c5-4156-9e27-a1befa67cae3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-cf05f599-c5c5-4156-9e27-a1befa67cae3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:01.156421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:01.157463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 11:37:01.158350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:01.160894 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 346/717] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:00 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:01.166835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c682f48-0d88-40cf-a205-5ca4f5e584f7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:01.185184 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:01.282901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c682f48-0d88-40cf-a205-5ca4f5e584f7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:01.318040 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db968cf4-da7d-4bd4-9772-21ec7ed414a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:01.319508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db968cf4-da7d-4bd4-9772-21ec7ed414a6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:01.319981 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db968cf4-da7d-4bd4-9772-21ec7ed414a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:01.320457 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db968cf4-da7d-4bd4-9772-21ec7ed414a6 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:01.321280 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db968cf4-da7d-4bd4-9772-21ec7ed414a6 None admin] RBAC: Adding query filter params (name=network) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:01.321788 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db968cf4-da7d-4bd4-9772-21ec7ed414a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:01.323404 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db968cf4-da7d-4bd4-9772-21ec7ed414a6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:01.328979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 372/718] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:01.334822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05f599-c5c5-4156-9e27-a1befa67cae3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:01.452547 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:01.461844 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05f599-c5c5-4156-9e27-a1befa67cae3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:01.506632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbce755-6151-43ef-a26e-8c5c5807ebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:01.507275 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbce755-6151-43ef-a26e-8c5c5807ebfb None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:01.507550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbce755-6151-43ef-a26e-8c5c5807ebfb None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:01.507812 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbce755-6151-43ef-a26e-8c5c5807ebfb None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:01.508452 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbce755-6151-43ef-a26e-8c5c5807ebfb None admin] RBAC: Adding query filter params (type=network) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:01.508749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbce755-6151-43ef-a26e-8c5c5807ebfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:01.510112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbce755-6151-43ef-a26e-8c5c5807ebfb None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:01.519302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 347/719] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:01.656614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d5777-4174-4c19-a94d-22aebb52cb4c None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:01.656793 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d5777-4174-4c19-a94d-22aebb52cb4c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:01.657069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d5777-4174-4c19-a94d-22aebb52cb4c None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:01.658651 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 373/720] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:01.664540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4271-0c0a-437f-aaf0-2467a822f493 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:01.664818 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4271-0c0a-437f-aaf0-2467a822f493 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:01.665094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4271-0c0a-437f-aaf0-2467a822f493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:01.692467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-83db4271-0c0a-437f-aaf0-2467a822f493 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:01.693700 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-83db4271-0c0a-437f-aaf0-2467a822f493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:01.762642 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 348/721] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:37:01.909373 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd46a9-11e9-492f-8c20-b7c7ad30fdee None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:01.909560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd46a9-11e9-492f-8c20-b7c7ad30fdee None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:01.909818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd46a9-11e9-492f-8c20-b7c7ad30fdee None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:01.911573 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 374/722] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:01.916867 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f53146-30a4-432c-ae67-5f0839e49aad None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:01.917148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f53146-30a4-432c-ae67-5f0839e49aad None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:01.917418 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f53146-30a4-432c-ae67-5f0839e49aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:01.948098 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-d4f53146-30a4-432c-ae67-5f0839e49aad None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:01.949276 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-d4f53146-30a4-432c-ae67-5f0839e49aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:02.036904 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 349/723] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:37:02.114095 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd46a9-11e9-492f-8c20-b7c7ad30fdee None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:02.221300 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:02.248194 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd46a9-11e9-492f-8c20-b7c7ad30fdee None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:02.282571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c6c37-4959-4263-9869-c73f6dcd13f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:02.283565 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c6c37-4959-4263-9869-c73f6dcd13f8 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:02.284033 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c6c37-4959-4263-9869-c73f6dcd13f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:02.284492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c6c37-4959-4263-9869-c73f6dcd13f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:02.295252 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c6c37-4959-4263-9869-c73f6dcd13f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-335c6c37-4959-4263-9869-c73f6dcd13f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:02.301676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:02.302595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:02.303469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:02.306035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 375/724] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:02.311635 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f53146-30a4-432c-ae67-5f0839e49aad None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:02.347553 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:02.388137 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c6c37-4959-4263-9869-c73f6dcd13f8 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:02.474068 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f53146-30a4-432c-ae67-5f0839e49aad None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:02.547677 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c6c37-4959-4263-9869-c73f6dcd13f8 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:02.590286 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5dc7e-bebf-4a6b-a39d-3f953a8a8468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:02.590930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5dc7e-bebf-4a6b-a39d-3f953a8a8468 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:02.591235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5dc7e-bebf-4a6b-a39d-3f953a8a8468 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:02.591498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5dc7e-bebf-4a6b-a39d-3f953a8a8468 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:02.592556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3356f-3008-4411-bd10-fa94d6289e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:02.593218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3356f-3008-4411-bd10-fa94d6289e13 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:02.593493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3356f-3008-4411-bd10-fa94d6289e13 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:02.593760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3356f-3008-4411-bd10-fa94d6289e13 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:02.594663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3356f-3008-4411-bd10-fa94d6289e13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:02.594962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3356f-3008-4411-bd10-fa94d6289e13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:02.597620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3356f-3008-4411-bd10-fa94d6289e13 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:02.604484 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:02.616301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5dc7e-bebf-4a6b-a39d-3f953a8a8468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:02.618266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 350/725] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:37:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:02.622318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3356f-3008-4411-bd10-fa94d6289e13 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-fca5dc7e-bebf-4a6b-a39d-3f953a8a8468 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:02.624502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:02.625409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:02.626243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 376/726] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:37:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:02.632050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5dc7e-bebf-4a6b-a39d-3f953a8a8468 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:02.785855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:02.814299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5dc7e-bebf-4a6b-a39d-3f953a8a8468 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:02.821783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3356f-3008-4411-bd10-fa94d6289e13 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:02.873349 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a226a-fb85-4b91-8147-f97c80eb4982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:02.873993 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a226a-fb85-4b91-8147-f97c80eb4982 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:02.874313 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a226a-fb85-4b91-8147-f97c80eb4982 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:02.874590 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a226a-fb85-4b91-8147-f97c80eb4982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:02.885678 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a226a-fb85-4b91-8147-f97c80eb4982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:02.887174 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a226a-fb85-4b91-8147-f97c80eb4982 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:02.887847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5dec7-528f-4dc1-bcb7-10b4ee038f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:02.888479 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5dec7-528f-4dc1-bcb7-10b4ee038f59 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:02.888753 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5dec7-528f-4dc1-bcb7-10b4ee038f59 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:02.889021 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5dec7-528f-4dc1-bcb7-10b4ee038f59 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:02.889701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5dec7-528f-4dc1-bcb7-10b4ee038f59 None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:02.890007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5dec7-528f-4dc1-bcb7-10b4ee038f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:02.891415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5dec7-528f-4dc1-bcb7-10b4ee038f59 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-262a226a-fb85-4b91-8147-f97c80eb4982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:02.903267 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:02.904544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:02.905678 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:02.905678 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:02.905678 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:02.906993 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 377/727] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:37:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:02.907760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 351/728] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:02.913331 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5dec7-528f-4dc1-bcb7-10b4ee038f59 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:02.914472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a226a-fb85-4b91-8147-f97c80eb4982 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:02.998664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:03.047814 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a226a-fb85-4b91-8147-f97c80eb4982 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.051932 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5dec7-528f-4dc1-bcb7-10b4ee038f59 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.089014 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d6ee9-85e2-4972-9a57-db7f0bdfc8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:03.089918 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d6ee9-85e2-4972-9a57-db7f0bdfc8df None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.090389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d6ee9-85e2-4972-9a57-db7f0bdfc8df None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.090881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d6ee9-85e2-4972-9a57-db7f0bdfc8df None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.091846 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d6ee9-85e2-4972-9a57-db7f0bdfc8df None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:03.092081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5eb16-da20-4c4d-91ee-3a7fc95f745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:03.092561 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d6ee9-85e2-4972-9a57-db7f0bdfc8df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:03.092729 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5eb16-da20-4c4d-91ee-3a7fc95f745f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.093034 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5eb16-da20-4c4d-91ee-3a7fc95f745f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.093306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5eb16-da20-4c4d-91ee-3a7fc95f745f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.094743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d6ee9-85e2-4972-9a57-db7f0bdfc8df None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:03.096877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:03.105984 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5eb16-da20-4c4d-91ee-3a7fc95f745f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:03.108226 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5eb16-da20-4c4d-91ee-3a7fc95f745f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:03.110314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 352/729] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:03.112167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a5ee2-a109-4cfe-a907-78ec723527d7 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.114435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a5ee2-a109-4cfe-a907-78ec723527d7 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.114714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a5ee2-a109-4cfe-a907-78ec723527d7 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.120282 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 353/730] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:03.121184 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a5ee2-a109-4cfe-a907-78ec723527d7 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-0fa5eb16-da20-4c4d-91ee-3a7fc95f745f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:03.126451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:03.127421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:03.128306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:03.128306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:03.128306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:03.130456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 378/731] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:03.136483 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d3737-a067-451d-b20f-f7a9f2af4ae0 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.136981 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d3737-a067-451d-b20f-f7a9f2af4ae0 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.137513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d3737-a067-451d-b20f-f7a9f2af4ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.157306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-a86d3737-a067-451d-b20f-f7a9f2af4ae0 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:03.158973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-a86d3737-a067-451d-b20f-f7a9f2af4ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:03.224308 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a5ee2-a109-4cfe-a907-78ec723527d7 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.235720 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 379/732] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:37:03.235720 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d3737-a067-451d-b20f-f7a9f2af4ae0 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:03.268116 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86c19b-722f-4885-86fb-10706f429adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:03.269007 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86c19b-722f-4885-86fb-10706f429adf None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.269286 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86c19b-722f-4885-86fb-10706f429adf None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.269554 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86c19b-722f-4885-86fb-10706f429adf None admin] PATH_INFO: `/v3/users/swift` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.279049 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86c19b-722f-4885-86fb-10706f429adf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:03.280556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86c19b-722f-4885-86fb-10706f429adf None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:03.285191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-bb86c19b-722f-4885-86fb-10706f429adf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:03.299084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:03.300295 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:03.301475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 354/733] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:03.302100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9314589-c6d6-4d6e-86ff-e7d517c73988 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.302100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9314589-c6d6-4d6e-86ff-e7d517c73988 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.302369 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9314589-c6d6-4d6e-86ff-e7d517c73988 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.304254 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 355/734] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:03.305926 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9314589-c6d6-4d6e-86ff-e7d517c73988 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:03.360479 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d3737-a067-451d-b20f-f7a9f2af4ae0 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.385818 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:03.406481 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7e97e-c34e-4def-b365-952bd8a46484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:03.408204 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7e97e-c34e-4def-b365-952bd8a46484 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.408622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7e97e-c34e-4def-b365-952bd8a46484 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.409028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7e97e-c34e-4def-b365-952bd8a46484 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.409816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7e97e-c34e-4def-b365-952bd8a46484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:03.410272 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7e97e-c34e-4def-b365-952bd8a46484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:03.412466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7e97e-c34e-4def-b365-952bd8a46484 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:03.432296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 380/735] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:03.432762 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3e0bf-4939-4947-b737-1915295b4c71 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.433170 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3e0bf-4939-4947-b737-1915295b4c71 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.433458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3e0bf-4939-4947-b737-1915295b4c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.450878 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9314589-c6d6-4d6e-86ff-e7d517c73988 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.456205 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-7af3e0bf-4939-4947-b737-1915295b4c71 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:03.457441 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-7af3e0bf-4939-4947-b737-1915295b4c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:03.487254 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31bed-a790-443b-a4c4-72219d26c7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:03.488224 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31bed-a790-443b-a4c4-72219d26c7ac None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.488737 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31bed-a790-443b-a4c4-72219d26c7ac None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.489205 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31bed-a790-443b-a4c4-72219d26c7ac None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.490136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31bed-a790-443b-a4c4-72219d26c7ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:03.490649 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31bed-a790-443b-a4c4-72219d26c7ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:03.492324 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31bed-a790-443b-a4c4-72219d26c7ac None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:03.510640 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 356/736] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Aug 30 11:37:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:03.511888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31bed-a790-443b-a4c4-72219d26c7ac None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:03.535147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 381/737] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:37:03.535147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3e0bf-4939-4947-b737-1915295b4c71 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:03.575693 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:03.617768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31bed-a790-443b-a4c4-72219d26c7ac None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.632165 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:03.657922 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5832745-35e3-4255-85ca-3e80dcdf25f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:03.658717 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832745-35e3-4255-85ca-3e80dcdf25f2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.659090 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832745-35e3-4255-85ca-3e80dcdf25f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.659476 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832745-35e3-4255-85ca-3e80dcdf25f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.661177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3e0bf-4939-4947-b737-1915295b4c71 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.667293 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5832745-35e3-4255-85ca-3e80dcdf25f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:03.668847 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5832745-35e3-4255-85ca-3e80dcdf25f2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-d5832745-35e3-4255-85ca-3e80dcdf25f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:03.681706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:03.682778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:03.683765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:03.688145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 357/738] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:37:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:03.689091 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5832745-35e3-4255-85ca-3e80dcdf25f2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:03.700702 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402533c6-1f24-4a63-bd65-b143738e38b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:03.701649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402533c6-1f24-4a63-bd65-b143738e38b0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.702213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402533c6-1f24-4a63-bd65-b143738e38b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.702676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402533c6-1f24-4a63-bd65-b143738e38b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.714049 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402533c6-1f24-4a63-bd65-b143738e38b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-402533c6-1f24-4a63-bd65-b143738e38b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:03.719530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:03.720448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:03.721281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:03.723991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 382/739] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:03.725079 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402533c6-1f24-4a63-bd65-b143738e38b0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:03.792943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5832745-35e3-4255-85ca-3e80dcdf25f2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.828367 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06000c-4ace-4371-8d91-4ac8296c8c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:03.829046 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:03.829197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06000c-4ace-4371-8d91-4ac8296c8c80 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.829427 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06000c-4ace-4371-8d91-4ac8296c8c80 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.829690 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06000c-4ace-4371-8d91-4ac8296c8c80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.838808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402533c6-1f24-4a63-bd65-b143738e38b0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.870904 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:03.871483 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde7b06-c95b-448a-a1ff-f797445ae522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:03.872130 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde7b06-c95b-448a-a1ff-f797445ae522 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.872383 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde7b06-c95b-448a-a1ff-f797445ae522 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.872588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde7b06-c95b-448a-a1ff-f797445ae522 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.873337 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde7b06-c95b-448a-a1ff-f797445ae522 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:03.873609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde7b06-c95b-448a-a1ff-f797445ae522 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:03.875037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde7b06-c95b-448a-a1ff-f797445ae522 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:03.892676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 383/740] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Aug 30 11:37:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:03.893921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2541dac-eb5e-4dfd-91c7-b1fa9db051aa None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:03.894207 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2541dac-eb5e-4dfd-91c7-b1fa9db051aa None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:03.894476 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2541dac-eb5e-4dfd-91c7-b1fa9db051aa None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:03.895760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 384/741] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:03.896448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2541dac-eb5e-4dfd-91c7-b1fa9db051aa None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:03.900995 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06000c-4ace-4371-8d91-4ac8296c8c80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:03.902450 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06000c-4ace-4371-8d91-4ac8296c8c80 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:03.996858 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2541dac-eb5e-4dfd-91c7-b1fa9db051aa None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:03.999996 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 358/742] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:37:04.000794 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06000c-4ace-4371-8d91-4ac8296c8c80 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:04.031417 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675047-9931-4726-837f-c46be2759dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:04.032016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675047-9931-4726-837f-c46be2759dda None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.032295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675047-9931-4726-837f-c46be2759dda None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.032549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675047-9931-4726-837f-c46be2759dda None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.033204 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675047-9931-4726-837f-c46be2759dda None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:04.033500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675047-9931-4726-837f-c46be2759dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:04.034884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675047-9931-4726-837f-c46be2759dda None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:04.042739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 385/743] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:37:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:04.043528 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675047-9931-4726-837f-c46be2759dda None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:04.055197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:04.073691 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:04.102109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06000c-4ace-4371-8d91-4ac8296c8c80 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:04.135512 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8a53b-03a7-4cf3-95fc-c408aac21440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:04.136106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8a53b-03a7-4cf3-95fc-c408aac21440 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.136376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8a53b-03a7-4cf3-95fc-c408aac21440 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.136647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8a53b-03a7-4cf3-95fc-c408aac21440 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.137291 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8a53b-03a7-4cf3-95fc-c408aac21440 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:04.138664 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8a53b-03a7-4cf3-95fc-c408aac21440 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:04.146225 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675047-9931-4726-837f-c46be2759dda None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:04.154646 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 359/744] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:37:03 2022] POST /identity/v3/services => generated 251 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:04.155854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6f5fd-6423-45ee-abba-1c35b3a9617b None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.156133 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6f5fd-6423-45ee-abba-1c35b3a9617b None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.156401 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6f5fd-6423-45ee-abba-1c35b3a9617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.185338 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-e8f6f5fd-6423-45ee-abba-1c35b3a9617b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:04.187008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-e8f6f5fd-6423-45ee-abba-1c35b3a9617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:04.194210 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32195d99-b0e8-4dcd-a908-19dd50e43402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:04.194910 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32195d99-b0e8-4dcd-a908-19dd50e43402 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.195232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32195d99-b0e8-4dcd-a908-19dd50e43402 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.195953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32195d99-b0e8-4dcd-a908-19dd50e43402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.263008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 360/745] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 11:37:04.263919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6f5fd-6423-45ee-abba-1c35b3a9617b None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:04.283169 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32195d99-b0e8-4dcd-a908-19dd50e43402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:04.285677 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32195d99-b0e8-4dcd-a908-19dd50e43402 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:04.308266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:04.346798 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:04.399861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6f5fd-6423-45ee-abba-1c35b3a9617b None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:04.412531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 386/746] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:04 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 11:37:04.413641 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32195d99-b0e8-4dcd-a908-19dd50e43402 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:04.448380 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e8dd3-202c-4f9d-a1f1-0e90188eebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:04.450073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e8dd3-202c-4f9d-a1f1-0e90188eebb5 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.450501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e8dd3-202c-4f9d-a1f1-0e90188eebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.450906 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e8dd3-202c-4f9d-a1f1-0e90188eebb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.462715 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e8dd3-202c-4f9d-a1f1-0e90188eebb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:04.464589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e8dd3-202c-4f9d-a1f1-0e90188eebb5 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-565e8dd3-202c-4f9d-a1f1-0e90188eebb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:04.479633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:04.480807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:04.481937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:04.481937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:04.481937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:04.483272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 361/747] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:04.484271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e8dd3-202c-4f9d-a1f1-0e90188eebb5 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:04.538292 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32195d99-b0e8-4dcd-a908-19dd50e43402 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:04.545913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:04.575871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4be85-1e62-4900-9bd4-9777657b7f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:04.576541 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4be85-1e62-4900-9bd4-9777657b7f42 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.576809 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4be85-1e62-4900-9bd4-9777657b7f42 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.577082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4be85-1e62-4900-9bd4-9777657b7f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.590428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e8dd3-202c-4f9d-a1f1-0e90188eebb5 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:04.591298 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4be85-1e62-4900-9bd4-9777657b7f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:04.594025 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4be85-1e62-4900-9bd4-9777657b7f42 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:04.606116 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-d6b4be85-1e62-4900-9bd4-9777657b7f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:04.611662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:04.612606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:04.618647 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:04.618647 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:04.618647 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:04.618647 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 387/748] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:04.618647 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4be85-1e62-4900-9bd4-9777657b7f42 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:04.635200 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c745-28e6-4d96-a07e-ba8723e194b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:04.635823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c745-28e6-4d96-a07e-ba8723e194b8 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.636095 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c745-28e6-4d96-a07e-ba8723e194b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.636495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c745-28e6-4d96-a07e-ba8723e194b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.647719 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634c745-28e6-4d96-a07e-ba8723e194b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:04.649172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634c745-28e6-4d96-a07e-ba8723e194b8 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-a634c745-28e6-4d96-a07e-ba8723e194b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:04.665687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:04.666644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:04.668060 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:04.668060 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:04.668060 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:04.668060 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 362/749] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:04.668060 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c745-28e6-4d96-a07e-ba8723e194b8 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:04.729628 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4be85-1e62-4900-9bd4-9777657b7f42 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:04.763257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e061d-ed54-45dc-84e1-6ba76cb05abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:04.763863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e061d-ed54-45dc-84e1-6ba76cb05abb None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.764139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e061d-ed54-45dc-84e1-6ba76cb05abb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.764401 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e061d-ed54-45dc-84e1-6ba76cb05abb None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.765035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e061d-ed54-45dc-84e1-6ba76cb05abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:04.765336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e061d-ed54-45dc-84e1-6ba76cb05abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:04.766721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e061d-ed54-45dc-84e1-6ba76cb05abb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:04.772036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c745-28e6-4d96-a07e-ba8723e194b8 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:04.783065 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 388/750] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:04.783065 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e061d-ed54-45dc-84e1-6ba76cb05abb None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:04.800020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:04.811224 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a611090-2590-417a-a04f-42f4a2add881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:04.811833 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a611090-2590-417a-a04f-42f4a2add881 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.812107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a611090-2590-417a-a04f-42f4a2add881 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.812372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a611090-2590-417a-a04f-42f4a2add881 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.813023 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a611090-2590-417a-a04f-42f4a2add881 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:04.814397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a611090-2590-417a-a04f-42f4a2add881 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:04.830296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 363/751] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:37:04 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:04.831155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a611090-2590-417a-a04f-42f4a2add881 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:04.863146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:04.919517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e061d-ed54-45dc-84e1-6ba76cb05abb None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:04.962845 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a611090-2590-417a-a04f-42f4a2add881 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:04.967804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f2779-9aac-45c7-b279-bc6283633fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:04.968670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f2779-9aac-45c7-b279-bc6283633fd6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:04.969164 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f2779-9aac-45c7-b279-bc6283633fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:04.969632 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f2779-9aac-45c7-b279-bc6283633fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:04.970466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f2779-9aac-45c7-b279-bc6283633fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:04.970956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f2779-9aac-45c7-b279-bc6283633fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:04.973177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f2779-9aac-45c7-b279-bc6283633fd6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:04.987389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 389/752] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:04.991138 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f2779-9aac-45c7-b279-bc6283633fd6 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.006240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657101a-4be6-4305-ad8c-f1b6e539e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.006860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657101a-4be6-4305-ad8c-f1b6e539e366 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.007156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657101a-4be6-4305-ad8c-f1b6e539e366 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.007521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657101a-4be6-4305-ad8c-f1b6e539e366 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.008168 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657101a-4be6-4305-ad8c-f1b6e539e366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:05.008472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657101a-4be6-4305-ad8c-f1b6e539e366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.010174 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657101a-4be6-4305-ad8c-f1b6e539e366 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.023988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 364/753] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:05.024772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657101a-4be6-4305-ad8c-f1b6e539e366 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.088543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:05.120751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f2779-9aac-45c7-b279-bc6283633fd6 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.130454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:05.153600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657101a-4be6-4305-ad8c-f1b6e539e366 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.166053 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf5ac4-14dc-4fba-b658-7216439b3da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.167102 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf5ac4-14dc-4fba-b658-7216439b3da5 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.167402 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf5ac4-14dc-4fba-b658-7216439b3da5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.167945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf5ac4-14dc-4fba-b658-7216439b3da5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.176816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf5ac4-14dc-4fba-b658-7216439b3da5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.179037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf5ac4-14dc-4fba-b658-7216439b3da5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-0ebf5ac4-14dc-4fba-b658-7216439b3da5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:05.194387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:05.195352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:05.196253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:05.196253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:05.196253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:05.196253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:05.196253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:05.196253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:05.196253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:05.196253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:05.198651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546de02-d7fe-464a-b5cc-915dbb22cd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.199381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546de02-d7fe-464a-b5cc-915dbb22cd7b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.199580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546de02-d7fe-464a-b5cc-915dbb22cd7b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.199770 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 390/754] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:05.199862 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546de02-d7fe-464a-b5cc-915dbb22cd7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.200971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf5ac4-14dc-4fba-b658-7216439b3da5 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.218242 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546de02-d7fe-464a-b5cc-915dbb22cd7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.220142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546de02-d7fe-464a-b5cc-915dbb22cd7b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-a546de02-d7fe-464a-b5cc-915dbb22cd7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:05.236489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:05.237566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 365/755] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:05.239192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546de02-d7fe-464a-b5cc-915dbb22cd7b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.321105 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf5ac4-14dc-4fba-b658-7216439b3da5 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.347316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546de02-d7fe-464a-b5cc-915dbb22cd7b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.359304 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bcf66-dd71-4287-8572-aaf7458e6a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.359943 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bcf66-dd71-4287-8572-aaf7458e6a65 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.360259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bcf66-dd71-4287-8572-aaf7458e6a65 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.360477 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bcf66-dd71-4287-8572-aaf7458e6a65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.367509 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6bcf66-dd71-4287-8572-aaf7458e6a65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.368957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6bcf66-dd71-4287-8572-aaf7458e6a65 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.370659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-9b6bcf66-dd71-4287-8572-aaf7458e6a65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:05.381837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:05.382745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:05.383701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:05.383701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:05.383701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:05.383701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:05.383701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:05.383701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:05.383701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:05.383701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:05.387714 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 391/756] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:05.388375 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619152b-5670-4a7d-8e17-7073fd8b339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.388573 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bcf66-dd71-4287-8572-aaf7458e6a65 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.389508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619152b-5670-4a7d-8e17-7073fd8b339c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.389979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619152b-5670-4a7d-8e17-7073fd8b339c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.390488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619152b-5670-4a7d-8e17-7073fd8b339c None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.391462 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619152b-5670-4a7d-8e17-7073fd8b339c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:05.391957 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619152b-5670-4a7d-8e17-7073fd8b339c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.393648 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619152b-5670-4a7d-8e17-7073fd8b339c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.406626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 366/757] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:05.408310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619152b-5670-4a7d-8e17-7073fd8b339c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.413011 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:05.497279 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bcf66-dd71-4287-8572-aaf7458e6a65 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.508824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619152b-5670-4a7d-8e17-7073fd8b339c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.532428 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a61854-c4b4-46ae-a89b-3959eb7dcce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.533115 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a61854-c4b4-46ae-a89b-3959eb7dcce7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.533332 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a61854-c4b4-46ae-a89b-3959eb7dcce7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.533598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a61854-c4b4-46ae-a89b-3959eb7dcce7 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.534322 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a61854-c4b4-46ae-a89b-3959eb7dcce7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:05.534831 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a61854-c4b4-46ae-a89b-3959eb7dcce7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.536107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a61854-c4b4-46ae-a89b-3959eb7dcce7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.545628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a8bd4-1eed-42db-8397-ab1ee002fb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.546511 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a8bd4-1eed-42db-8397-ab1ee002fb95 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.547027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a8bd4-1eed-42db-8397-ab1ee002fb95 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.547532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a8bd4-1eed-42db-8397-ab1ee002fb95 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.548417 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a8bd4-1eed-42db-8397-ab1ee002fb95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:05.548929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a8bd4-1eed-42db-8397-ab1ee002fb95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.550552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a8bd4-1eed-42db-8397-ab1ee002fb95 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.554349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 392/758] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:05.555320 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a61854-c4b4-46ae-a89b-3959eb7dcce7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.566299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 367/759] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:05.567482 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a8bd4-1eed-42db-8397-ab1ee002fb95 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.620009 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:05.639477 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:05.667147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a61854-c4b4-46ae-a89b-3959eb7dcce7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.669697 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a8bd4-1eed-42db-8397-ab1ee002fb95 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.701926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d43995-7ee1-433e-89b6-b96906258daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.702456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d43995-7ee1-433e-89b6-b96906258daf None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.702728 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d43995-7ee1-433e-89b6-b96906258daf None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.702991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d43995-7ee1-433e-89b6-b96906258daf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.704046 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d43995-7ee1-433e-89b6-b96906258daf None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=8bbb8fb036684305bc00290c519939ec) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:05.704369 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d43995-7ee1-433e-89b6-b96906258daf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.705707 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652ed5b-f26f-44b9-8b40-776c5d61b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.705833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d43995-7ee1-433e-89b6-b96906258daf None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.706645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652ed5b-f26f-44b9-8b40-776c5d61b72c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.707185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652ed5b-f26f-44b9-8b40-776c5d61b72c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.707654 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652ed5b-f26f-44b9-8b40-776c5d61b72c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.719227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 393/760] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/role_assignments?user.id=8bbb8fb036684305bc00290c519939ec&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:05.720256 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d43995-7ee1-433e-89b6-b96906258daf None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.721886 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652ed5b-f26f-44b9-8b40-776c5d61b72c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.723802 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652ed5b-f26f-44b9-8b40-776c5d61b72c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-4652ed5b-f26f-44b9-8b40-776c5d61b72c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:05.739698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:05.740704 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:05.741600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:05.741600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:05.741600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:05.743852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 368/761] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:05.752671 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652ed5b-f26f-44b9-8b40-776c5d61b72c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:05.900701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652ed5b-f26f-44b9-8b40-776c5d61b72c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.916438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:05.938073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd372bb-6484-4413-91d1-7a4a724d19b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:05.939068 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd372bb-6484-4413-91d1-7a4a724d19b8 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:05.939533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd372bb-6484-4413-91d1-7a4a724d19b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:05.939962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd372bb-6484-4413-91d1-7a4a724d19b8 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:05.940747 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd372bb-6484-4413-91d1-7a4a724d19b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:05.941189 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd372bb-6484-4413-91d1-7a4a724d19b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:05.942765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd372bb-6484-4413-91d1-7a4a724d19b8 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:05.957084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 369/762] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:05.957293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d43995-7ee1-433e-89b6-b96906258daf None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:05.963894 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd372bb-6484-4413-91d1-7a4a724d19b8 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:06.003192 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:06.055829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca465b52-43bd-47bd-b816-42671262dd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:06.056457 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca465b52-43bd-47bd-b816-42671262dd04 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:06.056727 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca465b52-43bd-47bd-b816-42671262dd04 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:06.056992 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca465b52-43bd-47bd-b816-42671262dd04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:06.057852 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca465b52-43bd-47bd-b816-42671262dd04 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=ee318396a8a04174a509a5faa8b86aa1) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:06.058017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca465b52-43bd-47bd-b816-42671262dd04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:06.059491 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca465b52-43bd-47bd-b816-42671262dd04 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:06.070579 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd372bb-6484-4413-91d1-7a4a724d19b8 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:06.073757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 394/763] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/role_assignments?user.id=ee318396a8a04174a509a5faa8b86aa1&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 259 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:06.099980 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933c7d5-2a93-4a06-9f9a-7e161e50a84d None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:06.100408 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933c7d5-2a93-4a06-9f9a-7e161e50a84d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:06.100827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933c7d5-2a93-4a06-9f9a-7e161e50a84d None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:06.102684 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 395/764] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:06.108023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871993-b828-4a9a-b31b-2e58e7e0af9e None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:06.108295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871993-b828-4a9a-b31b-2e58e7e0af9e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:06.108603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871993-b828-4a9a-b31b-2e58e7e0af9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:06.113031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad1264-0cba-4aa4-932f-91341970e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:06.113995 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad1264-0cba-4aa4-932f-91341970e4e6 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:06.114416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad1264-0cba-4aa4-932f-91341970e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:06.114879 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad1264-0cba-4aa4-932f-91341970e4e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:06.122884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad1264-0cba-4aa4-932f-91341970e4e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:06.124821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad1264-0cba-4aa4-932f-91341970e4e6 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-9dad1264-0cba-4aa4-932f-91341970e4e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:06.137744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:06.138940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:06.140194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:06.140194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:06.140194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:06.140194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:06.140194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:06.140194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:06.140194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:06.140194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:06.142359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 370/765] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:06.148131 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad1264-0cba-4aa4-932f-91341970e4e6 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:06.150035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-0c871993-b828-4a9a-b31b-2e58e7e0af9e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:06.159104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-0c871993-b828-4a9a-b31b-2e58e7e0af9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:06.218603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:06.284050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 396/766] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:06.311244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad1264-0cba-4aa4-932f-91341970e4e6 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:06.354280 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ab6a9-8730-4f6e-9cb1-d0bad0bc23df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:06.355170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ab6a9-8730-4f6e-9cb1-d0bad0bc23df None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:06.355613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ab6a9-8730-4f6e-9cb1-d0bad0bc23df None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:06.356040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ab6a9-8730-4f6e-9cb1-d0bad0bc23df None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:06.356884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ab6a9-8730-4f6e-9cb1-d0bad0bc23df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:06.357330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ab6a9-8730-4f6e-9cb1-d0bad0bc23df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:06.358890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ab6a9-8730-4f6e-9cb1-d0bad0bc23df None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:06.376948 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 371/767] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:06.388260 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c871993-b828-4a9a-b31b-2e58e7e0af9e None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:06.538349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c871993-b828-4a9a-b31b-2e58e7e0af9e None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:06.558384 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:06.559330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ab6a9-8730-4f6e-9cb1-d0bad0bc23df None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:06.596370 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934555c-56c7-402c-a181-9f88623bc061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:06.597035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934555c-56c7-402c-a181-9f88623bc061 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:06.597306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934555c-56c7-402c-a181-9f88623bc061 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:06.597568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934555c-56c7-402c-a181-9f88623bc061 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/0bb80229c3234d5f9599b16870c69934/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:06.600717 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934555c-56c7-402c-a181-9f88623bc061 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=0bb80229c3234d5f9599b16870c69934, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:06.602151 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934555c-56c7-402c-a181-9f88623bc061 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:06.625267 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 397/768] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:37:06 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/0bb80229c3234d5f9599b16870c69934/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:37:06.687596 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:06.696070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ab6a9-8730-4f6e-9cb1-d0bad0bc23df None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:06.751186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adece29a-4646-4e44-99a8-6c48b089d4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:06.752004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adece29a-4646-4e44-99a8-6c48b089d4c5 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:06.752281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adece29a-4646-4e44-99a8-6c48b089d4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:06.752550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adece29a-4646-4e44-99a8-6c48b089d4c5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:06.753142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adece29a-4646-4e44-99a8-6c48b089d4c5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:06.754721 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adece29a-4646-4e44-99a8-6c48b089d4c5 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-adece29a-4646-4e44-99a8-6c48b089d4c5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:06.764606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:06.765669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:06.766521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 372/769] 10.222.0.16 () {56 vars in 1129 bytes} [Tue Aug 30 11:37:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:06.771855 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934555c-56c7-402c-a181-9f88623bc061 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:06.891742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934555c-56c7-402c-a181-9f88623bc061 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:06.927537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99505e-f4e0-49cf-b85a-5c39a087097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:06.928207 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99505e-f4e0-49cf-b85a-5c39a087097b None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:06.928482 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99505e-f4e0-49cf-b85a-5c39a087097b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:06.928748 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99505e-f4e0-49cf-b85a-5c39a087097b None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:06.929381 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99505e-f4e0-49cf-b85a-5c39a087097b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:06.929682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99505e-f4e0-49cf-b85a-5c39a087097b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:06.931487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99505e-f4e0-49cf-b85a-5c39a087097b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:06.936926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 398/770] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Aug 30 11:37:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:06.943372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adece29a-4646-4e44-99a8-6c48b089d4c5 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:07.060497 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa16d13-c8ed-44e9-a4ea-be68ffb2e4dc None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:07.061145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa16d13-c8ed-44e9-a4ea-be68ffb2e4dc None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:07.061613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa16d13-c8ed-44e9-a4ea-be68ffb2e4dc None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:07.063529 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 399/771] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:07.070198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1cdc0-f1f1-4ac2-86a4-7b6e545bdd7d None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:07.070498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1cdc0-f1f1-4ac2-86a4-7b6e545bdd7d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:07.070739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1cdc0-f1f1-4ac2-86a4-7b6e545bdd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:07.091275 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adece29a-4646-4e44-99a8-6c48b089d4c5 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:07.093659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-1dd1cdc0-f1f1-4ac2-86a4-7b6e545bdd7d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:07.094869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-1dd1cdc0-f1f1-4ac2-86a4-7b6e545bdd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:07.155290 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bd2a4-ac5b-4711-ba60-24bf3fa3ea80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:07.155503 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:07.155820 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bd2a4-ac5b-4711-ba60-24bf3fa3ea80 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:07.156093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bd2a4-ac5b-4711-ba60-24bf3fa3ea80 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:07.156360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bd2a4-ac5b-4711-ba60-24bf3fa3ea80 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:07.157837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bd2a4-ac5b-4711-ba60-24bf3fa3ea80 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:07.158715 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bd2a4-ac5b-4711-ba60-24bf3fa3ea80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:07.163560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bd2a4-ac5b-4711-ba60-24bf3fa3ea80 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:07.176951 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 373/772] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Aug 30 11:37:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:07.182702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bd2a4-ac5b-4711-ba60-24bf3fa3ea80 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:07.209088 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:07.238205 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 400/773] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:07.290681 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bd2a4-ac5b-4711-ba60-24bf3fa3ea80 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:07.324764 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309d623-1ffb-4f21-9b04-5059a146601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:07.325373 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309d623-1ffb-4f21-9b04-5059a146601c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:07.325645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309d623-1ffb-4f21-9b04-5059a146601c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:07.325911 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309d623-1ffb-4f21-9b04-5059a146601c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:07.326619 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309d623-1ffb-4f21-9b04-5059a146601c None admin] RBAC: Adding query filter params (interface=public, service_id=76d1d4790a5c4e14ad87b3c9acaaafae, region_id=RegionOne) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:07.326919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309d623-1ffb-4f21-9b04-5059a146601c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:07.328366 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309d623-1ffb-4f21-9b04-5059a146601c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:07.336613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 374/774] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Aug 30 11:37:07 2022] GET /identity/v3/endpoints?service_id=76d1d4790a5c4e14ad87b3c9acaaafae&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:07.342108 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1cdc0-f1f1-4ac2-86a4-7b6e545bdd7d None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:07.444454 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1cdc0-f1f1-4ac2-86a4-7b6e545bdd7d None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:07.481766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83151deb-4e63-47eb-9d6e-3142769a09c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:07.482380 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83151deb-4e63-47eb-9d6e-3142769a09c6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:07.482651 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83151deb-4e63-47eb-9d6e-3142769a09c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:07.482913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83151deb-4e63-47eb-9d6e-3142769a09c6 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:07.483543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83151deb-4e63-47eb-9d6e-3142769a09c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:07.484974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83151deb-4e63-47eb-9d6e-3142769a09c6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:07.490692 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 401/775] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Aug 30 11:37:07 2022] GET /identity/v3/services => generated 772 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:07.670818 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309d623-1ffb-4f21-9b04-5059a146601c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:07.760298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:07.777334 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309d623-1ffb-4f21-9b04-5059a146601c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:07.811863 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7b977-1aad-4d0b-bd97-640418507c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:07.812489 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7b977-1aad-4d0b-bd97-640418507c9b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:07.812762 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7b977-1aad-4d0b-bd97-640418507c9b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:07.813026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7b977-1aad-4d0b-bd97-640418507c9b None admin] PATH_INFO: `/v3/services/network` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:07.813610 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7b977-1aad-4d0b-bd97-640418507c9b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:07.814971 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7b977-1aad-4d0b-bd97-640418507c9b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-27d7b977-1aad-4d0b-bd97-640418507c9b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:07.823889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:07.824866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:07.825741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 375/776] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:07 2022] GET /identity/v3/services/network => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:07.831611 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83151deb-4e63-47eb-9d6e-3142769a09c6 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:07.874789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:07.958772 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83151deb-4e63-47eb-9d6e-3142769a09c6 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:07.994608 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae3514-a44d-43cf-b562-adf8e1d9ff4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:07.995252 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae3514-a44d-43cf-b562-adf8e1d9ff4b None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:07.995523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae3514-a44d-43cf-b562-adf8e1d9ff4b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:07.995785 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae3514-a44d-43cf-b562-adf8e1d9ff4b None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:07.996472 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae3514-a44d-43cf-b562-adf8e1d9ff4b None admin] RBAC: Adding query filter params (name=network) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:07.996767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae3514-a44d-43cf-b562-adf8e1d9ff4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:07.998197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae3514-a44d-43cf-b562-adf8e1d9ff4b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:08.003260 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 402/777] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:08.012540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7b977-1aad-4d0b-bd97-640418507c9b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:08.065137 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b38d-4cd5-4f99-876e-34cd141fbb74 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:08.065492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b38d-4cd5-4f99-876e-34cd141fbb74 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:08.065682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b38d-4cd5-4f99-876e-34cd141fbb74 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:08.067435 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 403/778] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:08.074261 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a84d6-48a2-4601-8763-9adf726a0350 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:08.074542 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a84d6-48a2-4601-8763-9adf726a0350 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:08.074815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a84d6-48a2-4601-8763-9adf726a0350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:08.104040 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-f17a84d6-48a2-4601-8763-9adf726a0350 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:08.106591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f17a84d6-48a2-4601-8763-9adf726a0350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:08.173521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7b977-1aad-4d0b-bd97-640418507c9b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:08.239685 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 404/779] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:08.246884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e389c-4a2d-4935-82e2-fbc9aae31e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:08.247537 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e389c-4a2d-4935-82e2-fbc9aae31e47 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:08.247813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e389c-4a2d-4935-82e2-fbc9aae31e47 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:08.248081 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e389c-4a2d-4935-82e2-fbc9aae31e47 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:08.248725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e389c-4a2d-4935-82e2-fbc9aae31e47 None admin] RBAC: Adding query filter params (type=network) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:08.249025 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e389c-4a2d-4935-82e2-fbc9aae31e47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:08.250408 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e389c-4a2d-4935-82e2-fbc9aae31e47 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:08.255755 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 376/780] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:08.261567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a84d6-48a2-4601-8763-9adf726a0350 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:08.295609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:08.336957 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07f16-ae7a-4ee0-8830-c444bf9040d5 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:08.337298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07f16-ae7a-4ee0-8830-c444bf9040d5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:08.337541 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07f16-ae7a-4ee0-8830-c444bf9040d5 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:08.339097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 377/781] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:08.356731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41ba33-3ecf-4e5d-b6b4-921a2bb03236 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:08.357013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41ba33-3ecf-4e5d-b6b4-921a2bb03236 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:08.357287 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41ba33-3ecf-4e5d-b6b4-921a2bb03236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:08.382913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a84d6-48a2-4601-8763-9adf726a0350 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:08.399095 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-db41ba33-3ecf-4e5d-b6b4-921a2bb03236 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:08.400320 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-db41ba33-3ecf-4e5d-b6b4-921a2bb03236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:08.426114 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4936e-d65d-4fcb-9714-dae94fbbfa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:08.426740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4936e-d65d-4fcb-9714-dae94fbbfa19 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:08.426939 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:08.427039 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4936e-d65d-4fcb-9714-dae94fbbfa19 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:08.427294 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4936e-d65d-4fcb-9714-dae94fbbfa19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:08.428029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4936e-d65d-4fcb-9714-dae94fbbfa19 None admin] RBAC: Adding query filter params (interface=public, service_id=c29cfefe2d97479b88717663a2c2460a, region_id=RegionOne) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:08.428299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4936e-d65d-4fcb-9714-dae94fbbfa19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:08.429781 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4936e-d65d-4fcb-9714-dae94fbbfa19 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:08.447799 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 405/782] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Aug 30 11:37:08 2022] GET /identity/v3/endpoints?service_id=c29cfefe2d97479b88717663a2c2460a&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:08.452324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4936e-d65d-4fcb-9714-dae94fbbfa19 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:08.528311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 378/783] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:08.576179 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:08.586218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41ba33-3ecf-4e5d-b6b4-921a2bb03236 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:08.598450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4936e-d65d-4fcb-9714-dae94fbbfa19 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:08.660101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d255e7-7ba1-46b9-9c8d-357fc9e9b332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:08.660782 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d255e7-7ba1-46b9-9c8d-357fc9e9b332 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:08.661055 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d255e7-7ba1-46b9-9c8d-357fc9e9b332 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:08.661322 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d255e7-7ba1-46b9-9c8d-357fc9e9b332 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:08.661928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d255e7-7ba1-46b9-9c8d-357fc9e9b332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:08.663329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d255e7-7ba1-46b9-9c8d-357fc9e9b332 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:08.673728 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 406/784] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Aug 30 11:37:08 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:08.771554 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41ba33-3ecf-4e5d-b6b4-921a2bb03236 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:08.819548 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51d30d-ffc8-42a0-93a0-4c78d3433f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:08.820183 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51d30d-ffc8-42a0-93a0-4c78d3433f98 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:08.820454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51d30d-ffc8-42a0-93a0-4c78d3433f98 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:08.820718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51d30d-ffc8-42a0-93a0-4c78d3433f98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:08.837480 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51d30d-ffc8-42a0-93a0-4c78d3433f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-6e51d30d-ffc8-42a0-93a0-4c78d3433f98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:08.842247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:08.843198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:08.844048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:08.847005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 379/785] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:08.852257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d255e7-7ba1-46b9-9c8d-357fc9e9b332 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:08.870299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51d30d-ffc8-42a0-93a0-4c78d3433f98 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:08.875694 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:08.976864 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51d30d-ffc8-42a0-93a0-4c78d3433f98 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:09.014807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d255e7-7ba1-46b9-9c8d-357fc9e9b332 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:09.015135 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d873ce-1d28-44c7-9117-1c4c4eef6c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:09.015801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d873ce-1d28-44c7-9117-1c4c4eef6c0b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.016076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d873ce-1d28-44c7-9117-1c4c4eef6c0b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.016347 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d873ce-1d28-44c7-9117-1c4c4eef6c0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.027240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d873ce-1d28-44c7-9117-1c4c4eef6c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-f6d873ce-1d28-44c7-9117-1c4c4eef6c0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:09.032874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:09.033817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:37:09.035108 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:09.038748 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 380/786] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:37:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:09.039713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d873ce-1d28-44c7-9117-1c4c4eef6c0b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:09.058167 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8ea19-36b9-4b33-a9dc-004190a7b85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:09.059318 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8ea19-36b9-4b33-a9dc-004190a7b85f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.059866 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8ea19-36b9-4b33-a9dc-004190a7b85f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.060326 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8ea19-36b9-4b33-a9dc-004190a7b85f None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.061186 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8ea19-36b9-4b33-a9dc-004190a7b85f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:09.061682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8ea19-36b9-4b33-a9dc-004190a7b85f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:09.063389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8ea19-36b9-4b33-a9dc-004190a7b85f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:09.077444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 407/787] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:37:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:09.079305 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8ea19-36b9-4b33-a9dc-004190a7b85f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:09.099390 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:09.105669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:09.155966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d873ce-1d28-44c7-9117-1c4c4eef6c0b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:09.192148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8ea19-36b9-4b33-a9dc-004190a7b85f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:09.202645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adcd47-4e16-42b5-ba32-d06231b6caeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:09.204313 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adcd47-4e16-42b5-ba32-d06231b6caeb None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.204884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adcd47-4e16-42b5-ba32-d06231b6caeb None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.205331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adcd47-4e16-42b5-ba32-d06231b6caeb None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.206423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adcd47-4e16-42b5-ba32-d06231b6caeb None admin] RBAC: Adding query filter params (name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:09.207423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adcd47-4e16-42b5-ba32-d06231b6caeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:09.209244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adcd47-4e16-42b5-ba32-d06231b6caeb None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:09.225293 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 381/788] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:37:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:09.227386 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adcd47-4e16-42b5-ba32-d06231b6caeb None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:09.262573 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaddbf-5bc4-4492-8960-8ff642df4bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:09.263305 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaddbf-5bc4-4492-8960-8ff642df4bef None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.263580 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaddbf-5bc4-4492-8960-8ff642df4bef None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.263844 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaddbf-5bc4-4492-8960-8ff642df4bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.279334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaddbf-5bc4-4492-8960-8ff642df4bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:09.280913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaddbf-5bc4-4492-8960-8ff642df4bef None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-4eaaddbf-5bc4-4492-8960-8ff642df4bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:09.296120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:09.297096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:09.298066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:09.298066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:09.298066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:09.302094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 408/789] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:09.303418 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad513b-1190-4f5a-b832-fe8878cadddd None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.303759 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad513b-1190-4f5a-b832-fe8878cadddd None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.304024 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad513b-1190-4f5a-b832-fe8878cadddd None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.305584 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 409/790] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:09.306277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad513b-1190-4f5a-b832-fe8878cadddd None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:09.350318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adcd47-4e16-42b5-ba32-d06231b6caeb None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:09.385393 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:09.386879 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:09.398910 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae89ef3-449d-41ea-91ae-cdd91e1fd89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:09.400206 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae89ef3-449d-41ea-91ae-cdd91e1fd89c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.400948 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae89ef3-449d-41ea-91ae-cdd91e1fd89c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.401423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae89ef3-449d-41ea-91ae-cdd91e1fd89c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.415759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae89ef3-449d-41ea-91ae-cdd91e1fd89c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:09.418116 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae89ef3-449d-41ea-91ae-cdd91e1fd89c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:09.422843 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad513b-1190-4f5a-b832-fe8878cadddd None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-9ae89ef3-449d-41ea-91ae-cdd91e1fd89c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:09.436778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:09.437768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:09.444339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:09.444339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:09.444339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:09.444339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 382/791] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:09.444339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff166f0-fa47-4957-b34a-81dd657acb85 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.444339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff166f0-fa47-4957-b34a-81dd657acb85 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.444339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff166f0-fa47-4957-b34a-81dd657acb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.466960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df1b50-dc29-43fa-bfb8-9278e625bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:09.467606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df1b50-dc29-43fa-bfb8-9278e625bd52 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.467878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df1b50-dc29-43fa-bfb8-9278e625bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.468135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df1b50-dc29-43fa-bfb8-9278e625bd52 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.468770 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df1b50-dc29-43fa-bfb8-9278e625bd52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:09.469059 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df1b50-dc29-43fa-bfb8-9278e625bd52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:09.470055 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-eff166f0-fa47-4957-b34a-81dd657acb85 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:09.470520 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df1b50-dc29-43fa-bfb8-9278e625bd52 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:09.471262 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-eff166f0-fa47-4957-b34a-81dd657acb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:09.484236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 410/792] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:09.485136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df1b50-dc29-43fa-bfb8-9278e625bd52 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:09.567230 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 383/793] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:09.569876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff166f0-fa47-4957-b34a-81dd657acb85 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:09.610102 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df1b50-dc29-43fa-bfb8-9278e625bd52 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:09.657807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:09.666948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d3f60-eb21-4f17-8419-b2949a063b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:09.667664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d3f60-eb21-4f17-8419-b2949a063b9d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.668472 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d3f60-eb21-4f17-8419-b2949a063b9d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.668808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d3f60-eb21-4f17-8419-b2949a063b9d None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.669651 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d3f60-eb21-4f17-8419-b2949a063b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:09.670015 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d3f60-eb21-4f17-8419-b2949a063b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:09.671474 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d3f60-eb21-4f17-8419-b2949a063b9d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:09.684410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 411/794] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:09.694300 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d3f60-eb21-4f17-8419-b2949a063b9d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:09.758864 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff166f0-fa47-4957-b34a-81dd657acb85 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:09.771241 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:09.802049 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc458c-8425-43f3-80e3-13b3ad34cadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:09.802777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc458c-8425-43f3-80e3-13b3ad34cadd None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.802937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc458c-8425-43f3-80e3-13b3ad34cadd None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.803232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc458c-8425-43f3-80e3-13b3ad34cadd None admin] PATH_INFO: `/v3/users/swift` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.811778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc458c-8425-43f3-80e3-13b3ad34cadd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:09.814027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc458c-8425-43f3-80e3-13b3ad34cadd None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-bfdc458c-8425-43f3-80e3-13b3ad34cadd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:09.829071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:09.831239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 384/795] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dc39c-8978-42e0-8b6f-d962892b4fed None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.833539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dc39c-8978-42e0-8b6f-d962892b4fed None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.837943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dc39c-8978-42e0-8b6f-d962892b4fed None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.838032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d3f60-eb21-4f17-8419-b2949a063b9d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:09.838122 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 385/796] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:09.843135 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817d000-3d6f-4ed3-a329-31fe29268da1 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.843135 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817d000-3d6f-4ed3-a329-31fe29268da1 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.843135 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817d000-3d6f-4ed3-a329-31fe29268da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.860718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-f817d000-3d6f-4ed3-a329-31fe29268da1 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:09.862234 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-f817d000-3d6f-4ed3-a329-31fe29268da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:09.876648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26aadae-bff9-4b52-9350-9fc99186b693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:09.877306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26aadae-bff9-4b52-9350-9fc99186b693 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:09.877550 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26aadae-bff9-4b52-9350-9fc99186b693 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:09.877816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26aadae-bff9-4b52-9350-9fc99186b693 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:09.885489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26aadae-bff9-4b52-9350-9fc99186b693 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:09.886926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26aadae-bff9-4b52-9350-9fc99186b693 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-c26aadae-bff9-4b52-9350-9fc99186b693 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:09.899253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:09.900177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:09.901107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:09.905148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 412/797] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:37:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:09.905943 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26aadae-bff9-4b52-9350-9fc99186b693 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:09.938376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 386/798] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:09.943273 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817d000-3d6f-4ed3-a329-31fe29268da1 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:09.961613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:10.027411 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26aadae-bff9-4b52-9350-9fc99186b693 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.061201 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:10.067177 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817d000-3d6f-4ed3-a329-31fe29268da1 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.069180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559d0ef-5b56-4ed7-9e70-36e1478f3c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.069801 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559d0ef-5b56-4ed7-9e70-36e1478f3c90 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.070833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559d0ef-5b56-4ed7-9e70-36e1478f3c90 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.071146 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559d0ef-5b56-4ed7-9e70-36e1478f3c90 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.071890 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559d0ef-5b56-4ed7-9e70-36e1478f3c90 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:10.072195 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559d0ef-5b56-4ed7-9e70-36e1478f3c90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.073645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559d0ef-5b56-4ed7-9e70-36e1478f3c90 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.096133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 413/799] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Aug 30 11:37:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:10.097080 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559d0ef-5b56-4ed7-9e70-36e1478f3c90 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.106350 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71119f4a-04c3-421f-8c0e-d00ee0bc814f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.107537 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71119f4a-04c3-421f-8c0e-d00ee0bc814f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.108019 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71119f4a-04c3-421f-8c0e-d00ee0bc814f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.108492 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71119f4a-04c3-421f-8c0e-d00ee0bc814f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.120654 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71119f4a-04c3-421f-8c0e-d00ee0bc814f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-71119f4a-04c3-421f-8c0e-d00ee0bc814f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:10.126052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:10.127073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:10.128300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:10.130791 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 387/800] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:10.131845 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71119f4a-04c3-421f-8c0e-d00ee0bc814f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.232208 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559d0ef-5b56-4ed7-9e70-36e1478f3c90 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.241037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:10.248972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71119f4a-04c3-421f-8c0e-d00ee0bc814f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.269181 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dfeeb-3bf2-4f30-9ec8-f874b5b486e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.269822 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dfeeb-3bf2-4f30-9ec8-f874b5b486e1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.270094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dfeeb-3bf2-4f30-9ec8-f874b5b486e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.270358 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dfeeb-3bf2-4f30-9ec8-f874b5b486e1 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.271073 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dfeeb-3bf2-4f30-9ec8-f874b5b486e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:10.271389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dfeeb-3bf2-4f30-9ec8-f874b5b486e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.272888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dfeeb-3bf2-4f30-9ec8-f874b5b486e1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.286076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e2170-9a3f-48fb-af4b-43590c84dceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.286706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e2170-9a3f-48fb-af4b-43590c84dceb None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.287001 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e2170-9a3f-48fb-af4b-43590c84dceb None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.287302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e2170-9a3f-48fb-af4b-43590c84dceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.291005 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 414/801] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:10.292075 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dfeeb-3bf2-4f30-9ec8-f874b5b486e1 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.297548 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e2170-9a3f-48fb-af4b-43590c84dceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.299027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e2170-9a3f-48fb-af4b-43590c84dceb None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-e06e2170-9a3f-48fb-af4b-43590c84dceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:10.313851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:10.314813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:10.315753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:10.315753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:10.315753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:10.315753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 388/802] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:10.315753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e2170-9a3f-48fb-af4b-43590c84dceb None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.328390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:10.405696 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dfeeb-3bf2-4f30-9ec8-f874b5b486e1 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.423222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e2170-9a3f-48fb-af4b-43590c84dceb None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.445973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03aa71-88d0-4653-bdb2-278635141689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.446587 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03aa71-88d0-4653-bdb2-278635141689 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.446867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03aa71-88d0-4653-bdb2-278635141689 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.447152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03aa71-88d0-4653-bdb2-278635141689 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.447826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03aa71-88d0-4653-bdb2-278635141689 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:10.448125 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03aa71-88d0-4653-bdb2-278635141689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.449508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03aa71-88d0-4653-bdb2-278635141689 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.458299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 415/803] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:10.459089 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03aa71-88d0-4653-bdb2-278635141689 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.466853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cdae3-a9fe-49f5-8e9a-4b332fff564d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.467537 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cdae3-a9fe-49f5-8e9a-4b332fff564d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.467813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cdae3-a9fe-49f5-8e9a-4b332fff564d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.468082 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cdae3-a9fe-49f5-8e9a-4b332fff564d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.468665 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cdae3-a9fe-49f5-8e9a-4b332fff564d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.470044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cdae3-a9fe-49f5-8e9a-4b332fff564d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-3c3cdae3-a9fe-49f5-8e9a-4b332fff564d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:10.479368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:10.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 11:37:10.481214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:10.485497 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 389/804] 10.222.0.16 () {56 vars in 1129 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:10.486346 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cdae3-a9fe-49f5-8e9a-4b332fff564d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.488404 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:10.569959 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03aa71-88d0-4653-bdb2-278635141689 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.579397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:10.605178 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de7d1d-01f9-434f-a318-660768b60add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.605808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de7d1d-01f9-434f-a318-660768b60add None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.606080 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de7d1d-01f9-434f-a318-660768b60add None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.606347 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de7d1d-01f9-434f-a318-660768b60add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.609326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cdae3-a9fe-49f5-8e9a-4b332fff564d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.617686 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de7d1d-01f9-434f-a318-660768b60add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.619151 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de7d1d-01f9-434f-a318-660768b60add None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-43de7d1d-01f9-434f-a318-660768b60add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:10.633616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:10.634617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:10.635544 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:10.635544 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:10.635544 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:10.639594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 416/805] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:10.640098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de7d1d-01f9-434f-a318-660768b60add None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.657019 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f064658-c14b-47de-99bf-fd73116ca10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.657635 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f064658-c14b-47de-99bf-fd73116ca10e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.657933 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f064658-c14b-47de-99bf-fd73116ca10e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.658180 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f064658-c14b-47de-99bf-fd73116ca10e None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.658826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f064658-c14b-47de-99bf-fd73116ca10e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:10.659156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f064658-c14b-47de-99bf-fd73116ca10e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.660599 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f064658-c14b-47de-99bf-fd73116ca10e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.674529 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 390/806] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:10.675382 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f064658-c14b-47de-99bf-fd73116ca10e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.731625 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:10.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de7d1d-01f9-434f-a318-660768b60add None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.782419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f064658-c14b-47de-99bf-fd73116ca10e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.797820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c4903-e4d3-408f-a1c0-1bf96400e1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.798730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c4903-e4d3-408f-a1c0-1bf96400e1d5 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.799008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c4903-e4d3-408f-a1c0-1bf96400e1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.799311 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c4903-e4d3-408f-a1c0-1bf96400e1d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.811773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c4903-e4d3-408f-a1c0-1bf96400e1d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.813369 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c4903-e4d3-408f-a1c0-1bf96400e1d5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.821608 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97b582-e95b-4d94-8c7d-1e30e5e2d700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.822219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97b582-e95b-4d94-8c7d-1e30e5e2d700 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.822490 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97b582-e95b-4d94-8c7d-1e30e5e2d700 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.822756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97b582-e95b-4d94-8c7d-1e30e5e2d700 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.823443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97b582-e95b-4d94-8c7d-1e30e5e2d700 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:10.823740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97b582-e95b-4d94-8c7d-1e30e5e2d700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.825103 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97b582-e95b-4d94-8c7d-1e30e5e2d700 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.826196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:10.830156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 391/807] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:10.830962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97b582-e95b-4d94-8c7d-1e30e5e2d700 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-f23c4903-e4d3-408f-a1c0-1bf96400e1d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:10.831671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:10.832745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:10.833616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:10.833616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:10.833616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:10.837579 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 417/808] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:10.838704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c4903-e4d3-408f-a1c0-1bf96400e1d5 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.939053 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97b582-e95b-4d94-8c7d-1e30e5e2d700 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.945396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c4903-e4d3-408f-a1c0-1bf96400e1d5 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:10.973098 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16227562-cd2b-4e6a-8d9b-3e18868d1c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.973706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16227562-cd2b-4e6a-8d9b-3e18868d1c7f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.973981 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16227562-cd2b-4e6a-8d9b-3e18868d1c7f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.974093 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:10.974240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16227562-cd2b-4e6a-8d9b-3e18868d1c7f None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.974874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16227562-cd2b-4e6a-8d9b-3e18868d1c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:10.975195 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16227562-cd2b-4e6a-8d9b-3e18868d1c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.976559 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16227562-cd2b-4e6a-8d9b-3e18868d1c7f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:10.980055 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67878efa-e963-4549-a8c5-387ce8d8bb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:10.980683 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67878efa-e963-4549-a8c5-387ce8d8bb98 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:10.980955 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67878efa-e963-4549-a8c5-387ce8d8bb98 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:10.981221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67878efa-e963-4549-a8c5-387ce8d8bb98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:10.987529 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 392/809] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:10.987760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67878efa-e963-4549-a8c5-387ce8d8bb98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:10.988331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16227562-cd2b-4e6a-8d9b-3e18868d1c7f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:10.989256 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67878efa-e963-4549-a8c5-387ce8d8bb98 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-67878efa-e963-4549-a8c5-387ce8d8bb98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:11.002828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:11.003997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 418/810] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:11.009917 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67878efa-e963-4549-a8c5-387ce8d8bb98 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.047923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:11.090650 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16227562-cd2b-4e6a-8d9b-3e18868d1c7f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.106300 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67878efa-e963-4549-a8c5-387ce8d8bb98 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.123563 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54fbeb-b30e-4660-86a1-4f7231c886f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.124197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54fbeb-b30e-4660-86a1-4f7231c886f2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.124470 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54fbeb-b30e-4660-86a1-4f7231c886f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.124734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54fbeb-b30e-4660-86a1-4f7231c886f2 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.125361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54fbeb-b30e-4660-86a1-4f7231c886f2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:11.125656 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54fbeb-b30e-4660-86a1-4f7231c886f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.127015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54fbeb-b30e-4660-86a1-4f7231c886f2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.131666 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 393/811] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Aug 30 11:37:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:11.132794 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec4501-2449-4663-a3e5-58c9017df568 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.133069 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec4501-2449-4663-a3e5-58c9017df568 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.133335 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec4501-2449-4663-a3e5-58c9017df568 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.134690 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 394/812] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:11.135235 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec4501-2449-4663-a3e5-58c9017df568 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.142695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6500df-898a-4131-8882-cbf8db0651f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.143345 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6500df-898a-4131-8882-cbf8db0651f9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.143619 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6500df-898a-4131-8882-cbf8db0651f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.143887 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6500df-898a-4131-8882-cbf8db0651f9 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.144527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6500df-898a-4131-8882-cbf8db0651f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:11.144822 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6500df-898a-4131-8882-cbf8db0651f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.146203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6500df-898a-4131-8882-cbf8db0651f9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.157955 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 419/813] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:11.158745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6500df-898a-4131-8882-cbf8db0651f9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.208945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:11.240649 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec4501-2449-4663-a3e5-58c9017df568 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.260517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6500df-898a-4131-8882-cbf8db0651f9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.263854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:11.275812 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f58dbb-fd3b-4eca-80f0-b72f2178ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.276434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f58dbb-fd3b-4eca-80f0-b72f2178ca02 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.276702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f58dbb-fd3b-4eca-80f0-b72f2178ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.276965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f58dbb-fd3b-4eca-80f0-b72f2178ca02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.284162 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f58dbb-fd3b-4eca-80f0-b72f2178ca02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.285759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f58dbb-fd3b-4eca-80f0-b72f2178ca02 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.297202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136b5ba-19ba-41de-9ccf-db0b48bae955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-62f58dbb-fd3b-4eca-80f0-b72f2178ca02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:11.297384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:11.298344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:11.299314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:11.299314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:11.299314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:11.299314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:11.299314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:11.299314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:11.299314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:11.299314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:11.299764 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b5ba-19ba-41de-9ccf-db0b48bae955 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.299764 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b5ba-19ba-41de-9ccf-db0b48bae955 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.299764 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b5ba-19ba-41de-9ccf-db0b48bae955 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.299942 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 395/814] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:11.300018 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136b5ba-19ba-41de-9ccf-db0b48bae955 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:11.300097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f58dbb-fd3b-4eca-80f0-b72f2178ca02 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.300168 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136b5ba-19ba-41de-9ccf-db0b48bae955 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.300755 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136b5ba-19ba-41de-9ccf-db0b48bae955 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.312771 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 420/815] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:11.313953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3be4c3-e765-4cd4-8e9a-d008b3869d60 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.314233 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3be4c3-e765-4cd4-8e9a-d008b3869d60 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.314703 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3be4c3-e765-4cd4-8e9a-d008b3869d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.332962 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-bd3be4c3-e765-4cd4-8e9a-d008b3869d60 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:11.334161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-bd3be4c3-e765-4cd4-8e9a-d008b3869d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:11.399528 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 421/816] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:11.400393 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3be4c3-e765-4cd4-8e9a-d008b3869d60 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.405783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f58dbb-fd3b-4eca-80f0-b72f2178ca02 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.441205 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613703f3-7f85-4e22-943f-a3d950fae3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.441850 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613703f3-7f85-4e22-943f-a3d950fae3b2 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.442122 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613703f3-7f85-4e22-943f-a3d950fae3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.442387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613703f3-7f85-4e22-943f-a3d950fae3b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.443053 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613703f3-7f85-4e22-943f-a3d950fae3b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.444486 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613703f3-7f85-4e22-943f-a3d950fae3b2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.460442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:11.467725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 396/817] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:37:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:11.468428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613703f3-7f85-4e22-943f-a3d950fae3b2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.516184 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:11.550567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3be4c3-e765-4cd4-8e9a-d008b3869d60 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.596086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a745d-7c7e-4207-9d16-a96de58b56dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.596809 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a745d-7c7e-4207-9d16-a96de58b56dc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.596980 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a745d-7c7e-4207-9d16-a96de58b56dc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.597249 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a745d-7c7e-4207-9d16-a96de58b56dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.605356 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a745d-7c7e-4207-9d16-a96de58b56dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.606956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a745d-7c7e-4207-9d16-a96de58b56dc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.610415 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613703f3-7f85-4e22-943f-a3d950fae3b2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-4e0a745d-7c7e-4207-9d16-a96de58b56dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:11.621238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:11.622199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 422/818] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Aug 30 11:37:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:11.623268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a745d-7c7e-4207-9d16-a96de58b56dc None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.660043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2616f99-5be1-4b3e-ab4e-a165d05a2036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.660724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2616f99-5be1-4b3e-ab4e-a165d05a2036 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.661002 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2616f99-5be1-4b3e-ab4e-a165d05a2036 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.661687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2616f99-5be1-4b3e-ab4e-a165d05a2036 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.662638 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2616f99-5be1-4b3e-ab4e-a165d05a2036 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:11.662956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2616f99-5be1-4b3e-ab4e-a165d05a2036 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.665254 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2616f99-5be1-4b3e-ab4e-a165d05a2036 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.678381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 397/819] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:11.679463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2616f99-5be1-4b3e-ab4e-a165d05a2036 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.730215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:11.736877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a745d-7c7e-4207-9d16-a96de58b56dc None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.771939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e382a4-4382-4d7c-aca4-d7019c7db0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.772605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e382a4-4382-4d7c-aca4-d7019c7db0a0 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.772879 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e382a4-4382-4d7c-aca4-d7019c7db0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.773149 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e382a4-4382-4d7c-aca4-d7019c7db0a0 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/8bbb8fb036684305bc00290c519939ec/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.775073 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e382a4-4382-4d7c-aca4-d7019c7db0a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=8bbb8fb036684305bc00290c519939ec, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.776556 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e382a4-4382-4d7c-aca4-d7019c7db0a0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.785163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:11.789564 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2616f99-5be1-4b3e-ab4e-a165d05a2036 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.799371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 423/820] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:37:11 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/8bbb8fb036684305bc00290c519939ec/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:37:11.799371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e382a4-4382-4d7c-aca4-d7019c7db0a0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.829885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ad2b7-b00b-4904-bb50-66bdc94679b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.830614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ad2b7-b00b-4904-bb50-66bdc94679b0 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.830781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ad2b7-b00b-4904-bb50-66bdc94679b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.831022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ad2b7-b00b-4904-bb50-66bdc94679b0 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.831790 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ad2b7-b00b-4904-bb50-66bdc94679b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:11.832125 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ad2b7-b00b-4904-bb50-66bdc94679b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.833559 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ad2b7-b00b-4904-bb50-66bdc94679b0 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.859169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 398/821] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Aug 30 11:37:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:11.860043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ad2b7-b00b-4904-bb50-66bdc94679b0 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:11.931905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e382a4-4382-4d7c-aca4-d7019c7db0a0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:11.978659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cc496-f472-4d57-ac2d-6fe87e06d5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:11.979581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cc496-f472-4d57-ac2d-6fe87e06d5f1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:11.979886 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cc496-f472-4d57-ac2d-6fe87e06d5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:11.980147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cc496-f472-4d57-ac2d-6fe87e06d5f1 None admin] PATH_INFO: `/v3/services/network` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:11.980783 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145cc496-f472-4d57-ac2d-6fe87e06d5f1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:11.982110 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145cc496-f472-4d57-ac2d-6fe87e06d5f1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-145cc496-f472-4d57-ac2d-6fe87e06d5f1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:11.999965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:12.001012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 11:37:12.001867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:12.002614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ad2b7-b00b-4904-bb50-66bdc94679b0 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:12.002689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 424/822] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:11 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:12.007265 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cc496-f472-4d57-ac2d-6fe87e06d5f1 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:12.030763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:12.065183 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e96fa4-553d-444a-8872-434e03a1e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:12.066093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e96fa4-553d-444a-8872-434e03a1e72e None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:12.066840 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e96fa4-553d-444a-8872-434e03a1e72e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:12.067059 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e96fa4-553d-444a-8872-434e03a1e72e None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/ee318396a8a04174a509a5faa8b86aa1/roles/0ee8ef3183064d76a7e90b3ae6c5376d` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:12.088674 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e96fa4-553d-444a-8872-434e03a1e72e None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=ee318396a8a04174a509a5faa8b86aa1, role_id=0ee8ef3183064d76a7e90b3ae6c5376d)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:12.090801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e96fa4-553d-444a-8872-434e03a1e72e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:12.119208 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 399/823] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:37:11 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/ee318396a8a04174a509a5faa8b86aa1/roles/0ee8ef3183064d76a7e90b3ae6c5376d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:37:12.120546 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e96fa4-553d-444a-8872-434e03a1e72e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:12.127568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cc496-f472-4d57-ac2d-6fe87e06d5f1 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:12.146388 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:12.203405 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875e9a2-335b-4c10-9836-40a17b056494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:12.204956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875e9a2-335b-4c10-9836-40a17b056494 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:12.206284 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875e9a2-335b-4c10-9836-40a17b056494 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:12.206751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875e9a2-335b-4c10-9836-40a17b056494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:12.312196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e96fa4-553d-444a-8872-434e03a1e72e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:12.353455 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3778b-ef64-4089-a854-7f0dab3f78d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:12.354063 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3778b-ef64-4089-a854-7f0dab3f78d7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:12.354335 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3778b-ef64-4089-a854-7f0dab3f78d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:12.354599 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3778b-ef64-4089-a854-7f0dab3f78d7 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:12.355258 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3778b-ef64-4089-a854-7f0dab3f78d7 None admin] RBAC: Adding query filter params (name=network) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:12.355554 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3778b-ef64-4089-a854-7f0dab3f78d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:12.356928 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3778b-ef64-4089-a854-7f0dab3f78d7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:12.362242 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 400/824] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:12.371571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3778b-ef64-4089-a854-7f0dab3f78d7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:12.374340 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875e9a2-335b-4c10-9836-40a17b056494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:12.375851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875e9a2-335b-4c10-9836-40a17b056494 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:12.402331 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:12.487794 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 425/825] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:37:12.500347 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875e9a2-335b-4c10-9836-40a17b056494 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:12.613490 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:12.647491 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3778b-ef64-4089-a854-7f0dab3f78d7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:12.655660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875e9a2-335b-4c10-9836-40a17b056494 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:12.686627 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d64770-3434-46f2-b05a-3f83914e7b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:12.687325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d64770-3434-46f2-b05a-3f83914e7b2f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:12.687604 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d64770-3434-46f2-b05a-3f83914e7b2f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:12.687872 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d64770-3434-46f2-b05a-3f83914e7b2f None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:12.688551 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d64770-3434-46f2-b05a-3f83914e7b2f None admin] RBAC: Adding query filter params (type=network) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:12.688851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d64770-3434-46f2-b05a-3f83914e7b2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:12.690585 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d64770-3434-46f2-b05a-3f83914e7b2f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:12.693487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:12.697949 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 401/826] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:12.704662 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d64770-3434-46f2-b05a-3f83914e7b2f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:12.755914 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a73f1a-50ee-484f-a2a6-c36f9a71d50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:12.756525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a73f1a-50ee-484f-a2a6-c36f9a71d50a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:12.756805 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a73f1a-50ee-484f-a2a6-c36f9a71d50a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:12.757076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a73f1a-50ee-484f-a2a6-c36f9a71d50a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:12.788563 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a73f1a-50ee-484f-a2a6-c36f9a71d50a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:12.790600 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a73f1a-50ee-484f-a2a6-c36f9a71d50a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-c4a73f1a-50ee-484f-a2a6-c36f9a71d50a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:12.825092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:12.826118 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:12.827041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:12.827041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:12.827041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:12.827041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 426/827] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:12.832979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d64770-3434-46f2-b05a-3f83914e7b2f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:12.847932 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a73f1a-50ee-484f-a2a6-c36f9a71d50a None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:12.882025 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdb747-db3c-4960-9421-7b2ceaecdf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:12.882603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdb747-db3c-4960-9421-7b2ceaecdf63 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:12.882879 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdb747-db3c-4960-9421-7b2ceaecdf63 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:12.883185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdb747-db3c-4960-9421-7b2ceaecdf63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:12.883880 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdb747-db3c-4960-9421-7b2ceaecdf63 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:12.885270 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdb747-db3c-4960-9421-7b2ceaecdf63 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:12.890236 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:12.913521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 402/828] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:37:12 2022] POST /identity/v3/endpoints => generated 337 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:13.046028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a73f1a-50ee-484f-a2a6-c36f9a71d50a None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:13.108537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e47d4b-1a2b-4f43-9ef1-6c091ef28524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:13.109605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e47d4b-1a2b-4f43-9ef1-6c091ef28524 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:13.110074 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e47d4b-1a2b-4f43-9ef1-6c091ef28524 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:13.110550 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e47d4b-1a2b-4f43-9ef1-6c091ef28524 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:13.111448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e47d4b-1a2b-4f43-9ef1-6c091ef28524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:13.111941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e47d4b-1a2b-4f43-9ef1-6c091ef28524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:13.113582 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e47d4b-1a2b-4f43-9ef1-6c091ef28524 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:13.136337 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 427/829] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:13.142188 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdb747-db3c-4960-9421-7b2ceaecdf63 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:13.272020 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdb747-db3c-4960-9421-7b2ceaecdf63 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:13.308443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a309e1-f96e-4f3c-96a6-4af191b98065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:13.309073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a309e1-f96e-4f3c-96a6-4af191b98065 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:13.309349 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a309e1-f96e-4f3c-96a6-4af191b98065 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:13.309614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a309e1-f96e-4f3c-96a6-4af191b98065 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:13.316480 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a309e1-f96e-4f3c-96a6-4af191b98065 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:13.318037 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a309e1-f96e-4f3c-96a6-4af191b98065 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-e3a309e1-f96e-4f3c-96a6-4af191b98065 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:13.331472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:13.332429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:13.333314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:13.333314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:13.333314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:13.333314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:13.333314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:13.333314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:13.333314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:13.333314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:13.333314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 403/830] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:13.344266 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e47d4b-1a2b-4f43-9ef1-6c091ef28524 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:13.383875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:13.463679 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e47d4b-1a2b-4f43-9ef1-6c091ef28524 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:13.498947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3e117-635a-41f3-984e-9e81dbbe3fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:13.499603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3e117-635a-41f3-984e-9e81dbbe3fef None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:13.499874 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3e117-635a-41f3-984e-9e81dbbe3fef None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:13.500139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3e117-635a-41f3-984e-9e81dbbe3fef None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:13.500830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3e117-635a-41f3-984e-9e81dbbe3fef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:13.501128 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3e117-635a-41f3-984e-9e81dbbe3fef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:13.502571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3e117-635a-41f3-984e-9e81dbbe3fef None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:13.515324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 428/831] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:13.521292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a309e1-f96e-4f3c-96a6-4af191b98065 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:13.576117 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:13.632265 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e042a-5fbb-4d16-b7cd-e561cae1b33e None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:13.632598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e042a-5fbb-4d16-b7cd-e561cae1b33e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:13.632875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e042a-5fbb-4d16-b7cd-e561cae1b33e None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:13.634380 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a309e1-f96e-4f3c-96a6-4af191b98065 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:13.634517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 429/832] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:13.639553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ff3e3-f32d-46ed-b45c-130d1c4857cc None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:13.640058 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ff3e3-f32d-46ed-b45c-130d1c4857cc None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:13.640517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ff3e3-f32d-46ed-b45c-130d1c4857cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:13.663152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-c88ff3e3-f32d-46ed-b45c-130d1c4857cc None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:13.665352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c88ff3e3-f32d-46ed-b45c-130d1c4857cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:13.669600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a29024-23cf-41fe-81ad-420fee446063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:13.670211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a29024-23cf-41fe-81ad-420fee446063 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:13.670486 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a29024-23cf-41fe-81ad-420fee446063 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:13.670750 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a29024-23cf-41fe-81ad-420fee446063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:13.671510 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a29024-23cf-41fe-81ad-420fee446063 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=0bb80229c3234d5f9599b16870c69934) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:13.671815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a29024-23cf-41fe-81ad-420fee446063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:13.673253 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a29024-23cf-41fe-81ad-420fee446063 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:13.681047 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 404/833] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:13 2022] GET /identity/v3/role_assignments?user.id=0bb80229c3234d5f9599b16870c69934&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:13.742236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 430/834] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:14.003562 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a29024-23cf-41fe-81ad-420fee446063 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:14.049246 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333afd9-55c3-4bf7-89f2-7fb31752b67a None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.049766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333afd9-55c3-4bf7-89f2-7fb31752b67a None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.049766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333afd9-55c3-4bf7-89f2-7fb31752b67a None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.051429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 431/835] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:14.056701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51699850-4b45-4174-8b77-7c4ae3462e31 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.056982 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51699850-4b45-4174-8b77-7c4ae3462e31 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.057254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51699850-4b45-4174-8b77-7c4ae3462e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.076626 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-51699850-4b45-4174-8b77-7c4ae3462e31 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:14.077826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-51699850-4b45-4174-8b77-7c4ae3462e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:14.092700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:14.117320 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a29024-23cf-41fe-81ad-420fee446063 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:14.135606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:14.157628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972f0f-4484-4729-8c80-d6ab441bc02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:14.158268 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972f0f-4484-4729-8c80-d6ab441bc02c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.158578 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972f0f-4484-4729-8c80-d6ab441bc02c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.158797 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972f0f-4484-4729-8c80-d6ab441bc02c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.159212 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 432/836] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:14.161743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2972f0f-4484-4729-8c80-d6ab441bc02c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:14.161973 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2972f0f-4484-4729-8c80-d6ab441bc02c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-e2972f0f-4484-4729-8c80-d6ab441bc02c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:14.174869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:14.175952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 11:37:14.176781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:14.179517 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 405/837] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:37:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:14.185532 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51699850-4b45-4174-8b77-7c4ae3462e31 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:14.288652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51699850-4b45-4174-8b77-7c4ae3462e31 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:14.323088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294072b8-7ada-4f7b-9627-39b6a700f553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:14.324086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294072b8-7ada-4f7b-9627-39b6a700f553 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.324584 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294072b8-7ada-4f7b-9627-39b6a700f553 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.325079 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294072b8-7ada-4f7b-9627-39b6a700f553 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.326009 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294072b8-7ada-4f7b-9627-39b6a700f553 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:14.326499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294072b8-7ada-4f7b-9627-39b6a700f553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:14.328095 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294072b8-7ada-4f7b-9627-39b6a700f553 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:14.333612 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 433/838] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:37:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:14.339544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972f0f-4484-4729-8c80-d6ab441bc02c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:14.410982 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072c19c-cf75-446b-9973-44fda243a415 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.411326 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072c19c-cf75-446b-9973-44fda243a415 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.411584 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072c19c-cf75-446b-9973-44fda243a415 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.414150 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 434/839] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:14.418293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24664f65-3a1b-4e0b-93f4-161e0559f560 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.418581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24664f65-3a1b-4e0b-93f4-161e0559f560 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.418828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24664f65-3a1b-4e0b-93f4-161e0559f560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.436751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-24664f65-3a1b-4e0b-93f4-161e0559f560 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:14.437934 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-24664f65-3a1b-4e0b-93f4-161e0559f560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:14.451483 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:14.466446 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972f0f-4484-4729-8c80-d6ab441bc02c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:14.502186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef2274-0450-4dc8-9ae0-2fa9947dbcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:14.502787 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef2274-0450-4dc8-9ae0-2fa9947dbcb4 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.503071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef2274-0450-4dc8-9ae0-2fa9947dbcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.503339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef2274-0450-4dc8-9ae0-2fa9947dbcb4 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.503986 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef2274-0450-4dc8-9ae0-2fa9947dbcb4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:14.504287 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef2274-0450-4dc8-9ae0-2fa9947dbcb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:14.505673 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef2274-0450-4dc8-9ae0-2fa9947dbcb4 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:14.515162 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 406/840] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:37:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:14.515162 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef2274-0450-4dc8-9ae0-2fa9947dbcb4 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:14.520396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 435/841] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:14.566041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:14.633786 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef2274-0450-4dc8-9ae0-2fa9947dbcb4 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:14.677584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b01e4b-1a6e-4df7-a1e3-6955861faa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:14.678235 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b01e4b-1a6e-4df7-a1e3-6955861faa1f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.678501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b01e4b-1a6e-4df7-a1e3-6955861faa1f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.678768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b01e4b-1a6e-4df7-a1e3-6955861faa1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.690816 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b01e4b-1a6e-4df7-a1e3-6955861faa1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-03b01e4b-1a6e-4df7-a1e3-6955861faa1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:14.696207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:14.697186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:14.698040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 407/842] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:14.703068 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24664f65-3a1b-4e0b-93f4-161e0559f560 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:14.797930 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b01e4b-1a6e-4df7-a1e3-6955861faa1f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:14.814506 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24664f65-3a1b-4e0b-93f4-161e0559f560 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:14.851044 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58dc38-c62a-497b-9faf-5c0dcb335338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:14.851702 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58dc38-c62a-497b-9faf-5c0dcb335338 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.851978 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58dc38-c62a-497b-9faf-5c0dcb335338 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.852244 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58dc38-c62a-497b-9faf-5c0dcb335338 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.852909 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58dc38-c62a-497b-9faf-5c0dcb335338 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:14.853209 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58dc38-c62a-497b-9faf-5c0dcb335338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:14.854703 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58dc38-c62a-497b-9faf-5c0dcb335338 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:14.863899 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 436/843] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:37:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:14.868991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58dc38-c62a-497b-9faf-5c0dcb335338 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:14.887471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:14.931253 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b01e4b-1a6e-4df7-a1e3-6955861faa1f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:14.933971 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:14.976351 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58dc38-c62a-497b-9faf-5c0dcb335338 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:14.980314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb5227-5992-45ec-bfe0-8409f907ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:14.981140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb5227-5992-45ec-bfe0-8409f907ef48 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:14.981563 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb5227-5992-45ec-bfe0-8409f907ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:14.981980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb5227-5992-45ec-bfe0-8409f907ef48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:14.997330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb5227-5992-45ec-bfe0-8409f907ef48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-5afb5227-5992-45ec-bfe0-8409f907ef48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:15.002808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:15.004274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:37:15.005119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:15.005751 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 408/844] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:37:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:15.017335 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb5227-5992-45ec-bfe0-8409f907ef48 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.025375 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77ff0-b001-4b12-88d7-f93f01434532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.025692 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77ff0-b001-4b12-88d7-f93f01434532 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.026027 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77ff0-b001-4b12-88d7-f93f01434532 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.026350 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77ff0-b001-4b12-88d7-f93f01434532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.038206 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a77ff0-b001-4b12-88d7-f93f01434532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.039714 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a77ff0-b001-4b12-88d7-f93f01434532 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-85a77ff0-b001-4b12-88d7-f93f01434532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:15.058505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:15.060002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:15.061352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:15.061352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:15.061352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:15.061352 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 437/845] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:15.065950 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77ff0-b001-4b12-88d7-f93f01434532 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.125131 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:15.146335 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb5227-5992-45ec-bfe0-8409f907ef48 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:15.173052 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77ff0-b001-4b12-88d7-f93f01434532 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:15.190339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86407a0c-55df-41ae-ac1b-0c252ac592ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.191701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86407a0c-55df-41ae-ac1b-0c252ac592ac None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.192175 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86407a0c-55df-41ae-ac1b-0c252ac592ac None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.192631 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86407a0c-55df-41ae-ac1b-0c252ac592ac None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.193483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86407a0c-55df-41ae-ac1b-0c252ac592ac None admin] RBAC: Adding query filter params (name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:15.194475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86407a0c-55df-41ae-ac1b-0c252ac592ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.196339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86407a0c-55df-41ae-ac1b-0c252ac592ac None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.208911 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 409/846] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:15.213637 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9033b1-a3d6-40e9-912e-6f4df21c54fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.214109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86407a0c-55df-41ae-ac1b-0c252ac592ac None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.214315 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9033b1-a3d6-40e9-912e-6f4df21c54fb None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.214591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9033b1-a3d6-40e9-912e-6f4df21c54fb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.214862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9033b1-a3d6-40e9-912e-6f4df21c54fb None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.215533 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9033b1-a3d6-40e9-912e-6f4df21c54fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:15.215827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9033b1-a3d6-40e9-912e-6f4df21c54fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.217203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9033b1-a3d6-40e9-912e-6f4df21c54fb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.238099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 438/847] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:15.239072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9033b1-a3d6-40e9-912e-6f4df21c54fb None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.247562 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:15.342609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86407a0c-55df-41ae-ac1b-0c252ac592ac None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:15.352447 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9033b1-a3d6-40e9-912e-6f4df21c54fb None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:15.370638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:15.387043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397e7fe-f17e-48ef-98bd-b413bf6ca5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.387993 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397e7fe-f17e-48ef-98bd-b413bf6ca5b2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.388446 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397e7fe-f17e-48ef-98bd-b413bf6ca5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.388910 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397e7fe-f17e-48ef-98bd-b413bf6ca5b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.392223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc0a91-19fe-47b9-950b-e703db6c2da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.393023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc0a91-19fe-47b9-950b-e703db6c2da7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.393437 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc0a91-19fe-47b9-950b-e703db6c2da7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.393842 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc0a91-19fe-47b9-950b-e703db6c2da7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.401779 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc0a91-19fe-47b9-950b-e703db6c2da7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.402909 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397e7fe-f17e-48ef-98bd-b413bf6ca5b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.403731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc0a91-19fe-47b9-950b-e703db6c2da7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.404707 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397e7fe-f17e-48ef-98bd-b413bf6ca5b2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-edbc0a91-19fe-47b9-950b-e703db6c2da7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:15.420780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:15.421791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 11:37:15.422732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-3397e7fe-f17e-48ef-98bd-b413bf6ca5b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:15.429741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:15.430714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:15.431636 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:15.431636 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:15.431636 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:15.432354 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 439/849] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:15.432590 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 410/849] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:15.433865 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc0a91-19fe-47b9-950b-e703db6c2da7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.435213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397e7fe-f17e-48ef-98bd-b413bf6ca5b2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.531777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:15.550988 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc0a91-19fe-47b9-950b-e703db6c2da7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:15.555348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397e7fe-f17e-48ef-98bd-b413bf6ca5b2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:15.588626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c1530-db87-4261-bf00-90424644c86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.589230 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c1530-db87-4261-bf00-90424644c86c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.589510 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c1530-db87-4261-bf00-90424644c86c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.589780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c1530-db87-4261-bf00-90424644c86c None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.590512 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c1530-db87-4261-bf00-90424644c86c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:15.590817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c1530-db87-4261-bf00-90424644c86c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.592464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c1530-db87-4261-bf00-90424644c86c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.593841 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2a80f-a01d-4dc4-9cb9-9e9234eb08d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.594653 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2a80f-a01d-4dc4-9cb9-9e9234eb08d2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.595772 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2a80f-a01d-4dc4-9cb9-9e9234eb08d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.596115 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2a80f-a01d-4dc4-9cb9-9e9234eb08d2 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.596818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2a80f-a01d-4dc4-9cb9-9e9234eb08d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:15.597181 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2a80f-a01d-4dc4-9cb9-9e9234eb08d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.598756 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2a80f-a01d-4dc4-9cb9-9e9234eb08d2 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.614795 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 411/851] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:15.615789 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c1530-db87-4261-bf00-90424644c86c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.617838 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 440/851] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:15.619069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2a80f-a01d-4dc4-9cb9-9e9234eb08d2 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.661368 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:15.718431 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c1530-db87-4261-bf00-90424644c86c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:15.732976 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2a80f-a01d-4dc4-9cb9-9e9234eb08d2 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:15.753090 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601a45a-dad8-4411-a619-b13e0e2479f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.753681 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601a45a-dad8-4411-a619-b13e0e2479f5 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.753977 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601a45a-dad8-4411-a619-b13e0e2479f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.754237 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601a45a-dad8-4411-a619-b13e0e2479f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.757198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:15.771335 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e25c6a-3750-40dd-9b83-d6c5e8e89154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.771988 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e25c6a-3750-40dd-9b83-d6c5e8e89154 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.772261 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e25c6a-3750-40dd-9b83-d6c5e8e89154 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.772525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e25c6a-3750-40dd-9b83-d6c5e8e89154 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.779493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e25c6a-3750-40dd-9b83-d6c5e8e89154 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.780929 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e25c6a-3750-40dd-9b83-d6c5e8e89154 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-c5e25c6a-3750-40dd-9b83-d6c5e8e89154 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:15.792291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:15.793231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:15.794173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 441/852] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:15.795263 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70248547-2a4e-4d9b-b27d-ea4e4b9cf0cd None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.795263 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70248547-2a4e-4d9b-b27d-ea4e4b9cf0cd None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.795441 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70248547-2a4e-4d9b-b27d-ea4e4b9cf0cd None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.796785 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 442/853] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:15.801539 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70248547-2a4e-4d9b-b27d-ea4e4b9cf0cd None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.830711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601a45a-dad8-4411-a619-b13e0e2479f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.832469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601a45a-dad8-4411-a619-b13e0e2479f5 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.906779 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:15.916721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70248547-2a4e-4d9b-b27d-ea4e4b9cf0cd None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:15.935156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 412/854] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:37:15.936464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dd75b-0fb9-42a2-9a8a-c8b5be7f33c9 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.936746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dd75b-0fb9-42a2-9a8a-c8b5be7f33c9 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.937014 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dd75b-0fb9-42a2-9a8a-c8b5be7f33c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.955720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-f39dd75b-0fb9-42a2-9a8a-c8b5be7f33c9 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:15.956960 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-f39dd75b-0fb9-42a2-9a8a-c8b5be7f33c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:15.961545 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28481c72-6ba1-4e67-b151-3ebadeed56c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:15.962156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28481c72-6ba1-4e67-b151-3ebadeed56c7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:15.962427 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28481c72-6ba1-4e67-b151-3ebadeed56c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:15.962689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28481c72-6ba1-4e67-b151-3ebadeed56c7 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:15.963440 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28481c72-6ba1-4e67-b151-3ebadeed56c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:15.963736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28481c72-6ba1-4e67-b151-3ebadeed56c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:15.965148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28481c72-6ba1-4e67-b151-3ebadeed56c7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:15.983843 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 443/855] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:15.984739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28481c72-6ba1-4e67-b151-3ebadeed56c7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:15.990030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:16.039154 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 413/856] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:16.039154 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dd75b-0fb9-42a2-9a8a-c8b5be7f33c9 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:16.092563 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28481c72-6ba1-4e67-b151-3ebadeed56c7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:16.128169 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a55435-d002-4984-8264-330c89f4f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:16.129031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a55435-d002-4984-8264-330c89f4f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.129311 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a55435-d002-4984-8264-330c89f4f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.129580 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a55435-d002-4984-8264-330c89f4f5d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.142960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a55435-d002-4984-8264-330c89f4f5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:16.145155 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a55435-d002-4984-8264-330c89f4f5d6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-48a55435-d002-4984-8264-330c89f4f5d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:16.164202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:16.165177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:16.166064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:16.166064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:16.166064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:16.170032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dd75b-0fb9-42a2-9a8a-c8b5be7f33c9 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:16.170463 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 444/857] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:16.171540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a55435-d002-4984-8264-330c89f4f5d6 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:16.183397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:16.213445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a8462-d7dc-454e-aff4-899785d3345e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:16.214053 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a8462-d7dc-454e-aff4-899785d3345e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.214628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a8462-d7dc-454e-aff4-899785d3345e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.214908 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a8462-d7dc-454e-aff4-899785d3345e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.254422 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:16.300984 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a8462-d7dc-454e-aff4-899785d3345e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:16.302468 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a8462-d7dc-454e-aff4-899785d3345e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:16.415066 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 414/858] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:16 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:37:16.415066 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a8462-d7dc-454e-aff4-899785d3345e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:16.428404 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a55435-d002-4984-8264-330c89f4f5d6 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:16.467062 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae61aca-4b9b-4428-ad6d-99da7996ff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:16.467765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae61aca-4b9b-4428-ad6d-99da7996ff94 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.468040 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae61aca-4b9b-4428-ad6d-99da7996ff94 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.468610 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae61aca-4b9b-4428-ad6d-99da7996ff94 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.469078 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae61aca-4b9b-4428-ad6d-99da7996ff94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:16.469378 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae61aca-4b9b-4428-ad6d-99da7996ff94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:16.470756 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae61aca-4b9b-4428-ad6d-99da7996ff94 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:16.477876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:16.482806 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 445/859] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:16.483652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae61aca-4b9b-4428-ad6d-99da7996ff94 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:16.515848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a8462-d7dc-454e-aff4-899785d3345e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:16.546022 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:16.551018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a219a-5ca4-41ae-8d14-38d29c7dead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:16.551667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a219a-5ca4-41ae-8d14-38d29c7dead8 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.551935 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a219a-5ca4-41ae-8d14-38d29c7dead8 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.552197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a219a-5ca4-41ae-8d14-38d29c7dead8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.552771 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a219a-5ca4-41ae-8d14-38d29c7dead8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:16.554132 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a219a-5ca4-41ae-8d14-38d29c7dead8 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-380a219a-5ca4-41ae-8d14-38d29c7dead8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:16.562709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:16.563849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:16.564923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 415/860] 10.222.0.16 () {56 vars in 1132 bytes} [Tue Aug 30 11:37:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:16.565997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8a12a-aa15-4eb7-8cd1-0a016f23daf7 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.565997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8a12a-aa15-4eb7-8cd1-0a016f23daf7 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.565997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8a12a-aa15-4eb7-8cd1-0a016f23daf7 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.567065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 416/861] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:16.567759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8a12a-aa15-4eb7-8cd1-0a016f23daf7 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:16.589548 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae61aca-4b9b-4428-ad6d-99da7996ff94 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:16.622877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200f51e-d188-40bb-bd6e-2ca257c9a96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:16.623537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200f51e-d188-40bb-bd6e-2ca257c9a96d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.623819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200f51e-d188-40bb-bd6e-2ca257c9a96d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.624079 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200f51e-d188-40bb-bd6e-2ca257c9a96d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.634155 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200f51e-d188-40bb-bd6e-2ca257c9a96d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:16.635929 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200f51e-d188-40bb-bd6e-2ca257c9a96d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-4200f51e-d188-40bb-bd6e-2ca257c9a96d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:16.649568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:16.650650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:16.651609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:16.651609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:16.651609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:16.651609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 446/862] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:16.651609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200f51e-d188-40bb-bd6e-2ca257c9a96d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:16.673116 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8a12a-aa15-4eb7-8cd1-0a016f23daf7 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:16.698660 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:16.706757 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6f49e-1f3b-4264-88e6-d99c28782e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:16.707465 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6f49e-1f3b-4264-88e6-d99c28782e2c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.707737 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6f49e-1f3b-4264-88e6-d99c28782e2c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.708003 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6f49e-1f3b-4264-88e6-d99c28782e2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.714818 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6f49e-1f3b-4264-88e6-d99c28782e2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:16.716305 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6f49e-1f3b-4264-88e6-d99c28782e2c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-dfa6f49e-1f3b-4264-88e6-d99c28782e2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:16.727318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:16.728359 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:16.729281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:16.729281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:16.729281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:16.729281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:16.729281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:16.729281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:16.729281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:16.729281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:16.729281 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 417/863] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:16.730203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028083d8-8a65-412d-8703-436c6b7fe008 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.730203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028083d8-8a65-412d-8703-436c6b7fe008 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.730377 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028083d8-8a65-412d-8703-436c6b7fe008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.747607 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-028083d8-8a65-412d-8703-436c6b7fe008 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:16.748753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-028083d8-8a65-412d-8703-436c6b7fe008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:16.754632 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200f51e-d188-40bb-bd6e-2ca257c9a96d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:16.771209 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:16.788446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5693e-a727-44d9-9d23-f4da9e3bcc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:16.789417 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5693e-a727-44d9-9d23-f4da9e3bcc15 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.789886 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5693e-a727-44d9-9d23-f4da9e3bcc15 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.790164 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5693e-a727-44d9-9d23-f4da9e3bcc15 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.791066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5693e-a727-44d9-9d23-f4da9e3bcc15 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:16.791628 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5693e-a727-44d9-9d23-f4da9e3bcc15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:16.793516 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5693e-a727-44d9-9d23-f4da9e3bcc15 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:16.798486 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 447/864] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:16.799282 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5693e-a727-44d9-9d23-f4da9e3bcc15 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:16.834494 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 418/865] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:16.835560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028083d8-8a65-412d-8703-436c6b7fe008 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:16.911468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5693e-a727-44d9-9d23-f4da9e3bcc15 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:16.941368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028083d8-8a65-412d-8703-436c6b7fe008 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:16.945612 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833bf30-112b-4ff2-97a5-c521f3aa3b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:16.946224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833bf30-112b-4ff2-97a5-c521f3aa3b82 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.946500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833bf30-112b-4ff2-97a5-c521f3aa3b82 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.946766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833bf30-112b-4ff2-97a5-c521f3aa3b82 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.947427 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833bf30-112b-4ff2-97a5-c521f3aa3b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:16.947724 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833bf30-112b-4ff2-97a5-c521f3aa3b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:16.949172 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833bf30-112b-4ff2-97a5-c521f3aa3b82 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:16.965963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 448/866] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:16.965963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833bf30-112b-4ff2-97a5-c521f3aa3b82 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:16.967759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:16.987857 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdcfbe-616d-4d84-9913-bed25c6038a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:16.988490 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdcfbe-616d-4d84-9913-bed25c6038a3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:16.988781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdcfbe-616d-4d84-9913-bed25c6038a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:16.989041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdcfbe-616d-4d84-9913-bed25c6038a3 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:16.989799 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdcfbe-616d-4d84-9913-bed25c6038a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:16.990006 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdcfbe-616d-4d84-9913-bed25c6038a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:16.991454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdcfbe-616d-4d84-9913-bed25c6038a3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:17.003694 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 419/867] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:17.004625 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdcfbe-616d-4d84-9913-bed25c6038a3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:17.012376 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:17.070863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833bf30-112b-4ff2-97a5-c521f3aa3b82 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:17.104937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a92766-10c4-4288-95b7-b13b9154e8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:17.105555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a92766-10c4-4288-95b7-b13b9154e8ee None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:17.105814 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a92766-10c4-4288-95b7-b13b9154e8ee None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:17.105910 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdcfbe-616d-4d84-9913-bed25c6038a3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:17.106271 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a92766-10c4-4288-95b7-b13b9154e8ee None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:17.106704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a92766-10c4-4288-95b7-b13b9154e8ee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:17.107005 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a92766-10c4-4288-95b7-b13b9154e8ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:17.108456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a92766-10c4-4288-95b7-b13b9154e8ee None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:17.113153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 449/868] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:17.113927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a92766-10c4-4288-95b7-b13b9154e8ee None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:17.143340 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef3c0c-5d51-4c19-9a15-752652faec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:17.144011 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef3c0c-5d51-4c19-9a15-752652faec89 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:17.144285 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef3c0c-5d51-4c19-9a15-752652faec89 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:17.144589 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef3c0c-5d51-4c19-9a15-752652faec89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:17.152341 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef3c0c-5d51-4c19-9a15-752652faec89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:17.153965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef3c0c-5d51-4c19-9a15-752652faec89 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-53ef3c0c-5d51-4c19-9a15-752652faec89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:17.166580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:17.167811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:17.169558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:17.169558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:17.169558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:17.169558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:17.169558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:17.169558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:17.169558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:17.169558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:17.169558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 420/869] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:17.171348 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef3c0c-5d51-4c19-9a15-752652faec89 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:17.217107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:17.236421 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a92766-10c4-4288-95b7-b13b9154e8ee None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:17.244907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:17.277877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb200e-7150-4ce9-a485-8f3c88df1801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:17.278471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb200e-7150-4ce9-a485-8f3c88df1801 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:17.278737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb200e-7150-4ce9-a485-8f3c88df1801 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:17.279000 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb200e-7150-4ce9-a485-8f3c88df1801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:17.279763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb200e-7150-4ce9-a485-8f3c88df1801 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=8bbb8fb036684305bc00290c519939ec) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:17.280059 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb200e-7150-4ce9-a485-8f3c88df1801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:17.281475 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb200e-7150-4ce9-a485-8f3c88df1801 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:17.286091 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef3c0c-5d51-4c19-9a15-752652faec89 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:17.287662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 450/870] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:17 2022] GET /identity/v3/role_assignments?user.id=8bbb8fb036684305bc00290c519939ec&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:17.288455 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb200e-7150-4ce9-a485-8f3c88df1801 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:17.348485 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628d4ce-6492-4d3d-ad40-fbfe9a872697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:17.349899 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628d4ce-6492-4d3d-ad40-fbfe9a872697 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:17.350556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628d4ce-6492-4d3d-ad40-fbfe9a872697 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:17.350998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628d4ce-6492-4d3d-ad40-fbfe9a872697 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:17.351840 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628d4ce-6492-4d3d-ad40-fbfe9a872697 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:17.353993 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628d4ce-6492-4d3d-ad40-fbfe9a872697 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:17.377036 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 421/871] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:37:17 2022] POST /identity/v3/services => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:17.420832 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb200e-7150-4ce9-a485-8f3c88df1801 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:17.470919 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc3bf1-f781-49e6-b42c-d2cdae28ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:17.471681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc3bf1-f781-49e6-b42c-d2cdae28ada9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:17.471952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc3bf1-f781-49e6-b42c-d2cdae28ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:17.472213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc3bf1-f781-49e6-b42c-d2cdae28ada9 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:17.472958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc3bf1-f781-49e6-b42c-d2cdae28ada9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:17.473256 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc3bf1-f781-49e6-b42c-d2cdae28ada9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:17.474736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc3bf1-f781-49e6-b42c-d2cdae28ada9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:17.489173 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 451/872] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:17.497032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628d4ce-6492-4d3d-ad40-fbfe9a872697 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:17.601440 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:17.605467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628d4ce-6492-4d3d-ad40-fbfe9a872697 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:17.640602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d4608-1700-4232-bdc1-a0fee567f805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:17.641209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d4608-1700-4232-bdc1-a0fee567f805 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:17.641478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d4608-1700-4232-bdc1-a0fee567f805 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:17.641740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d4608-1700-4232-bdc1-a0fee567f805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:17.642457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d4608-1700-4232-bdc1-a0fee567f805 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=ee318396a8a04174a509a5faa8b86aa1) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:17.642754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d4608-1700-4232-bdc1-a0fee567f805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:17.644269 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d4608-1700-4232-bdc1-a0fee567f805 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:17.650854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 422/873] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:17 2022] GET /identity/v3/role_assignments?user.id=ee318396a8a04174a509a5faa8b86aa1&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:19.040897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00cc5a-dbe2-4f86-a517-2369aafaed82 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.041498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00cc5a-dbe2-4f86-a517-2369aafaed82 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.041498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00cc5a-dbe2-4f86-a517-2369aafaed82 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.043012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 452/874] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:19.048425 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504555f0-cf56-40da-afaa-2c7d0e3c1543 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.048848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504555f0-cf56-40da-afaa-2c7d0e3c1543 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.049008 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504555f0-cf56-40da-afaa-2c7d0e3c1543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.067698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-504555f0-cf56-40da-afaa-2c7d0e3c1543 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:19.068897 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-504555f0-cf56-40da-afaa-2c7d0e3c1543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:19.135684 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 423/875] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:19.301434 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69313db-c25d-4b4d-bfd4-edf0a4a16497 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.301632 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69313db-c25d-4b4d-bfd4-edf0a4a16497 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.302028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69313db-c25d-4b4d-bfd4-edf0a4a16497 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.303757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 453/876] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:19.308947 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931ddb1-5bf2-413b-8970-6e9f73e6de7b None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.309296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931ddb1-5bf2-413b-8970-6e9f73e6de7b None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.309635 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931ddb1-5bf2-413b-8970-6e9f73e6de7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.327793 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-2931ddb1-5bf2-413b-8970-6e9f73e6de7b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:19.329266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-2931ddb1-5bf2-413b-8970-6e9f73e6de7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:19.333690 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb259db-340e-4348-baeb-69c45adc0cfa None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.333974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb259db-340e-4348-baeb-69c45adc0cfa None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.334242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb259db-340e-4348-baeb-69c45adc0cfa None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.335873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 454/877] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:19.341864 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529639f0-c915-4af3-b388-5896c001ad8b None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.342144 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529639f0-c915-4af3-b388-5896c001ad8b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.342412 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529639f0-c915-4af3-b388-5896c001ad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.360821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-529639f0-c915-4af3-b388-5896c001ad8b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:19.362098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-529639f0-c915-4af3-b388-5896c001ad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:19.384165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:19.410930 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 424/878] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:19.412219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931ddb1-5bf2-413b-8970-6e9f73e6de7b None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:19.443856 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:19.463251 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 455/879] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:19.522819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931ddb1-5bf2-413b-8970-6e9f73e6de7b None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:19.557095 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3c196-9c88-43aa-9e6b-6b50e866d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:19.557997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3c196-9c88-43aa-9e6b-6b50e866d638 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.558452 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3c196-9c88-43aa-9e6b-6b50e866d638 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.558908 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3c196-9c88-43aa-9e6b-6b50e866d638 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.559770 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3c196-9c88-43aa-9e6b-6b50e866d638 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:19.561310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3c196-9c88-43aa-9e6b-6b50e866d638 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:19.580520 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 425/880] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:37:19 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:37:19.592329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cd00c-f28c-4ee2-b328-5de5ce984105 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.592633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cd00c-f28c-4ee2-b328-5de5ce984105 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.592877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cd00c-f28c-4ee2-b328-5de5ce984105 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.594540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 456/881] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:19.600400 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a16f7f-62b1-43bd-a34b-48ef512db59c None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.600677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a16f7f-62b1-43bd-a34b-48ef512db59c None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.600948 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a16f7f-62b1-43bd-a34b-48ef512db59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.620248 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-e7a16f7f-62b1-43bd-a34b-48ef512db59c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:19.621501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-e7a16f7f-62b1-43bd-a34b-48ef512db59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:19.676732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cd00c-f28c-4ee2-b328-5de5ce984105 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:19.709006 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 426/882] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:19.749877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a16f7f-62b1-43bd-a34b-48ef512db59c None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:19.761058 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:19.810201 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cd00c-f28c-4ee2-b328-5de5ce984105 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:19.851518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb78f8-a30f-4f7f-83ba-cef84db82153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:19.852159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb78f8-a30f-4f7f-83ba-cef84db82153 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.852435 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb78f8-a30f-4f7f-83ba-cef84db82153 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.852701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb78f8-a30f-4f7f-83ba-cef84db82153 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.853282 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb78f8-a30f-4f7f-83ba-cef84db82153 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:19.854661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb78f8-a30f-4f7f-83ba-cef84db82153 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-5acb78f8-a30f-4f7f-83ba-cef84db82153 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:19.863789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:19.864804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:19.865660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 457/883] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:37:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:19.872399 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb78f8-a30f-4f7f-83ba-cef84db82153 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:19.881266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a16f7f-62b1-43bd-a34b-48ef512db59c None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:19.900868 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:19.929411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a1086-324d-422b-bb74-dfb0fd8968a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:19.930091 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a1086-324d-422b-bb74-dfb0fd8968a3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:19.930364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a1086-324d-422b-bb74-dfb0fd8968a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:19.930628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a1086-324d-422b-bb74-dfb0fd8968a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:19.931510 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a1086-324d-422b-bb74-dfb0fd8968a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:19.932745 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a1086-324d-422b-bb74-dfb0fd8968a3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-b87a1086-324d-422b-bb74-dfb0fd8968a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:19.942247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:19.943330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:19.944170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 427/884] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:19.949148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a1086-324d-422b-bb74-dfb0fd8968a3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:19.991174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb78f8-a30f-4f7f-83ba-cef84db82153 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.026885 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a30303-e613-43b7-a6b9-76f59b0a7225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.027535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a30303-e613-43b7-a6b9-76f59b0a7225 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.028298 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a30303-e613-43b7-a6b9-76f59b0a7225 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.028549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a30303-e613-43b7-a6b9-76f59b0a7225 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.029190 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a30303-e613-43b7-a6b9-76f59b0a7225 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:20.029496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a30303-e613-43b7-a6b9-76f59b0a7225 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:20.032447 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a30303-e613-43b7-a6b9-76f59b0a7225 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:20.033624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:20.038383 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 458/885] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:37:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:20.039405 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a30303-e613-43b7-a6b9-76f59b0a7225 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:20.070320 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a1086-324d-422b-bb74-dfb0fd8968a3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.108114 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c07654-bb8f-4cc9-acc2-aaa749e6b89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.108735 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c07654-bb8f-4cc9-acc2-aaa749e6b89c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.109001 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c07654-bb8f-4cc9-acc2-aaa749e6b89c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.109284 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c07654-bb8f-4cc9-acc2-aaa749e6b89c None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.109894 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c07654-bb8f-4cc9-acc2-aaa749e6b89c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:20.110196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c07654-bb8f-4cc9-acc2-aaa749e6b89c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:20.111612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c07654-bb8f-4cc9-acc2-aaa749e6b89c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:20.116511 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 428/886] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:20.117122 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c07654-bb8f-4cc9-acc2-aaa749e6b89c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:20.138799 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:20.156229 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a30303-e613-43b7-a6b9-76f59b0a7225 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.192916 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c1708-2c8e-4cd7-9fb2-b1bf9f3d5f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.193946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c1708-2c8e-4cd7-9fb2-b1bf9f3d5f13 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.194223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c1708-2c8e-4cd7-9fb2-b1bf9f3d5f13 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.194489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c1708-2c8e-4cd7-9fb2-b1bf9f3d5f13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.205433 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c1708-2c8e-4cd7-9fb2-b1bf9f3d5f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-646c1708-2c8e-4cd7-9fb2-b1bf9f3d5f13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:20.210774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:20.211896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:20.212776 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 459/887] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Aug 30 11:37:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:20.213464 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c1708-2c8e-4cd7-9fb2-b1bf9f3d5f13 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:20.229373 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c07654-bb8f-4cc9-acc2-aaa749e6b89c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.264015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3c36-f1bc-4fe5-89b4-8068c3539850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.264613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3c36-f1bc-4fe5-89b4-8068c3539850 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.264885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3c36-f1bc-4fe5-89b4-8068c3539850 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.265173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3c36-f1bc-4fe5-89b4-8068c3539850 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.265763 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb3c36-f1bc-4fe5-89b4-8068c3539850 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:20.266057 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb3c36-f1bc-4fe5-89b4-8068c3539850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:20.267613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb3c36-f1bc-4fe5-89b4-8068c3539850 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:20.269761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:20.278253 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 429/888] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:37:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:20.278895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3c36-f1bc-4fe5-89b4-8068c3539850 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:20.322464 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c1708-2c8e-4cd7-9fb2-b1bf9f3d5f13 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.360144 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01e2d9-4414-4f5c-8f88-16584d005cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.360756 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01e2d9-4414-4f5c-8f88-16584d005cde None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.361030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01e2d9-4414-4f5c-8f88-16584d005cde None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.361301 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01e2d9-4414-4f5c-8f88-16584d005cde None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.361923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01e2d9-4414-4f5c-8f88-16584d005cde None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:20.362223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01e2d9-4414-4f5c-8f88-16584d005cde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:20.363688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01e2d9-4414-4f5c-8f88-16584d005cde None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:20.371165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 460/889] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:20.371165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01e2d9-4414-4f5c-8f88-16584d005cde None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:20.390217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:20.404056 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3c36-f1bc-4fe5-89b4-8068c3539850 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.453464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaf301-16c8-4c4d-9c8e-f28320b1ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.454768 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaf301-16c8-4c4d-9c8e-f28320b1ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.455044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaf301-16c8-4c4d-9c8e-f28320b1ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.455355 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaf301-16c8-4c4d-9c8e-f28320b1ccf2 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.456029 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeaf301-16c8-4c4d-9c8e-f28320b1ccf2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:20.456328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeaf301-16c8-4c4d-9c8e-f28320b1ccf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:20.457799 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeaf301-16c8-4c4d-9c8e-f28320b1ccf2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:20.471293 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 430/890] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Aug 30 11:37:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:20.473769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaf301-16c8-4c4d-9c8e-f28320b1ccf2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:20.514779 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01e2d9-4414-4f5c-8f88-16584d005cde None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.547885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:20.553242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c0fc9-f2ab-4e59-8f79-cd80d71508fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.554389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c0fc9-f2ab-4e59-8f79-cd80d71508fd None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.554389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c0fc9-f2ab-4e59-8f79-cd80d71508fd None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.554389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c0fc9-f2ab-4e59-8f79-cd80d71508fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.555082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c0fc9-f2ab-4e59-8f79-cd80d71508fd None admin] RBAC: Adding query filter params (interface=public, service_id=9e0ad47c2e9d48d18a7f8f591761d911, region_id=RegionOne) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:20.555426 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c0fc9-f2ab-4e59-8f79-cd80d71508fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:20.556894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c0fc9-f2ab-4e59-8f79-cd80d71508fd None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:20.563285 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 461/891] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Aug 30 11:37:20 2022] GET /identity/v3/endpoints?service_id=9e0ad47c2e9d48d18a7f8f591761d911&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:20.568851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c0fc9-f2ab-4e59-8f79-cd80d71508fd None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:20.594626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaf301-16c8-4c4d-9c8e-f28320b1ccf2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.630565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa937ace-41c2-4f5f-9034-9646dfdfb75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.631223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa937ace-41c2-4f5f-9034-9646dfdfb75e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.631493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa937ace-41c2-4f5f-9034-9646dfdfb75e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.631750 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa937ace-41c2-4f5f-9034-9646dfdfb75e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.645000 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa937ace-41c2-4f5f-9034-9646dfdfb75e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:20.646448 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa937ace-41c2-4f5f-9034-9646dfdfb75e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-fa937ace-41c2-4f5f-9034-9646dfdfb75e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:20.661210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:20.662209 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:20.663744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:20.663744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:20.663744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:20.663744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 431/892] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:20.666378 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:20.668924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa937ace-41c2-4f5f-9034-9646dfdfb75e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:20.709033 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c0fc9-f2ab-4e59-8f79-cd80d71508fd None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.753328 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a99da-0171-4375-af36-018b31414eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.754455 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a99da-0171-4375-af36-018b31414eae None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.754932 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a99da-0171-4375-af36-018b31414eae None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.755439 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a99da-0171-4375-af36-018b31414eae None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.756280 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a99da-0171-4375-af36-018b31414eae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:20.757907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a99da-0171-4375-af36-018b31414eae None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:20.767234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 462/893] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Aug 30 11:37:20 2022] GET /identity/v3/services => generated 1268 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 11:37:20.790793 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa937ace-41c2-4f5f-9034-9646dfdfb75e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:20.806918 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:20.833656 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bebb90-7bec-4c77-90b1-6c5ddde40aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:20.834640 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bebb90-7bec-4c77-90b1-6c5ddde40aec None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:20.835155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bebb90-7bec-4c77-90b1-6c5ddde40aec None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:20.835617 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bebb90-7bec-4c77-90b1-6c5ddde40aec None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:20.836604 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bebb90-7bec-4c77-90b1-6c5ddde40aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:20.837099 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bebb90-7bec-4c77-90b1-6c5ddde40aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:20.838719 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bebb90-7bec-4c77-90b1-6c5ddde40aec None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:20.855195 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 432/894] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:20.864262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a99da-0171-4375-af36-018b31414eae None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:20.973035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a99da-0171-4375-af36-018b31414eae None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:21.007969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797417f-4696-403f-ba61-05cf22542235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:21.008605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797417f-4696-403f-ba61-05cf22542235 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:21.008873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797417f-4696-403f-ba61-05cf22542235 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:21.009136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797417f-4696-403f-ba61-05cf22542235 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:21.012412 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:21.019038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797417f-4696-403f-ba61-05cf22542235 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:21.021138 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797417f-4696-403f-ba61-05cf22542235 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-a797417f-4696-403f-ba61-05cf22542235 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:21.033571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:21.034508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:21.036101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 463/895] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:37:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:21.040790 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bebb90-7bec-4c77-90b1-6c5ddde40aec None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:21.146296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bebb90-7bec-4c77-90b1-6c5ddde40aec None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:21.181624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a3e2b-a5b8-4dc8-954f-f6eb21d0f2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:21.182572 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a3e2b-a5b8-4dc8-954f-f6eb21d0f2f8 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:21.183064 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a3e2b-a5b8-4dc8-954f-f6eb21d0f2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:21.183539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a3e2b-a5b8-4dc8-954f-f6eb21d0f2f8 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:21.184472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a3e2b-a5b8-4dc8-954f-f6eb21d0f2f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:21.184979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a3e2b-a5b8-4dc8-954f-f6eb21d0f2f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:21.186603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a3e2b-a5b8-4dc8-954f-f6eb21d0f2f8 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:21.205406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 433/896] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Aug 30 11:37:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:21.211102 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797417f-4696-403f-ba61-05cf22542235 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:21.313972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797417f-4696-403f-ba61-05cf22542235 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:21.349343 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3ad11-1f52-442c-9e97-551b9615ae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:21.350315 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3ad11-1f52-442c-9e97-551b9615ae91 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:21.350779 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3ad11-1f52-442c-9e97-551b9615ae91 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:21.351281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3ad11-1f52-442c-9e97-551b9615ae91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:21.418010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:21.425021 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3ad11-1f52-442c-9e97-551b9615ae91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:21.426730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3ad11-1f52-442c-9e97-551b9615ae91 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:21.531992 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 464/897] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:21 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:37:21.537411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a3e2b-a5b8-4dc8-954f-f6eb21d0f2f8 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:21.568469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:21.589304 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ffc0f-ca5d-4322-9eb5-8bd849b321d0 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:21.589594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ffc0f-ca5d-4322-9eb5-8bd849b321d0 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:21.589857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ffc0f-ca5d-4322-9eb5-8bd849b321d0 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:21.592675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 465/898] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:21.597744 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2d693-cac4-4c45-9967-c75e0bc03f72 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:21.598163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2d693-cac4-4c45-9967-c75e0bc03f72 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:21.598583 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2d693-cac4-4c45-9967-c75e0bc03f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:21.620165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-16e2d693-cac4-4c45-9967-c75e0bc03f72 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:21.621662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-16e2d693-cac4-4c45-9967-c75e0bc03f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:21.640790 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a3e2b-a5b8-4dc8-954f-f6eb21d0f2f8 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:21.675150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e9195-5d05-47cb-b2e4-0791a6729bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:21.675924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e9195-5d05-47cb-b2e4-0791a6729bb2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:21.676127 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e9195-5d05-47cb-b2e4-0791a6729bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:21.676384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e9195-5d05-47cb-b2e4-0791a6729bb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:21.686662 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e9195-5d05-47cb-b2e4-0791a6729bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:21.688199 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e9195-5d05-47cb-b2e4-0791a6729bb2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:21.694176 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 466/899] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-684e9195-5d05-47cb-b2e4-0791a6729bb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:21.702965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:21.704387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:21.705299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:21.705299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:21.705299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:21.705299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 434/900] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:21.709994 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2d693-cac4-4c45-9967-c75e0bc03f72 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:21.737732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:21.811690 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2d693-cac4-4c45-9967-c75e0bc03f72 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:21.847072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aacaca-8ae2-4929-ad01-ceb6a133d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:21.848083 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aacaca-8ae2-4929-ad01-ceb6a133d652 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:21.848592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aacaca-8ae2-4929-ad01-ceb6a133d652 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:21.849076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aacaca-8ae2-4929-ad01-ceb6a133d652 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:21.849918 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aacaca-8ae2-4929-ad01-ceb6a133d652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:21.850419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aacaca-8ae2-4929-ad01-ceb6a133d652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:21.852021 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aacaca-8ae2-4929-ad01-ceb6a133d652 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:21.864239 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 467/901] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:21.871301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e9195-5d05-47cb-b2e4-0791a6729bb2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:21.955540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:21.959230 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aacaca-8ae2-4929-ad01-ceb6a133d652 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:21.972428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e9195-5d05-47cb-b2e4-0791a6729bb2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:22.014280 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaf9b7-6288-4e62-9243-84afce3e9625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:22.015388 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaf9b7-6288-4e62-9243-84afce3e9625 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.015681 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaf9b7-6288-4e62-9243-84afce3e9625 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.015988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaf9b7-6288-4e62-9243-84afce3e9625 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.025464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaf9b7-6288-4e62-9243-84afce3e9625 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:22.027513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaf9b7-6288-4e62-9243-84afce3e9625 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-3bfaf9b7-6288-4e62-9243-84afce3e9625 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:22.043651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:22.044565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:22.045830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:22.045830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:22.045830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:22.045830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:22.045830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:22.045830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:22.045830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:22.045830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:22.045830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 435/902] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:22.050390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaf9b7-6288-4e62-9243-84afce3e9625 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:22.063752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:22.080157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aacaca-8ae2-4929-ad01-ceb6a133d652 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:22.113787 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d606ad-1504-4abe-b790-170af4b1ab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:22.114734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d606ad-1504-4abe-b790-170af4b1ab8e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.115218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d606ad-1504-4abe-b790-170af4b1ab8e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.115715 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d606ad-1504-4abe-b790-170af4b1ab8e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.116458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d606ad-1504-4abe-b790-170af4b1ab8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:22.118063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d606ad-1504-4abe-b790-170af4b1ab8e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-a7d606ad-1504-4abe-b790-170af4b1ab8e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:22.127076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:22.128063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 11:37:22.129069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:22.131900 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 468/903] 10.222.0.16 () {56 vars in 1132 bytes} [Tue Aug 30 11:37:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:22.137663 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d606ad-1504-4abe-b790-170af4b1ab8e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:22.164312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaf9b7-6288-4e62-9243-84afce3e9625 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:22.200730 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b76cb-edf9-49ad-b070-29af6c5c3555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:22.201442 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b76cb-edf9-49ad-b070-29af6c5c3555 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.201759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b76cb-edf9-49ad-b070-29af6c5c3555 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.202096 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b76cb-edf9-49ad-b070-29af6c5c3555 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.202850 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b76cb-edf9-49ad-b070-29af6c5c3555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:22.203249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b76cb-edf9-49ad-b070-29af6c5c3555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:22.205150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b76cb-edf9-49ad-b070-29af6c5c3555 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:22.219368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 436/904] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:22.225227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b76cb-edf9-49ad-b070-29af6c5c3555 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:22.230461 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:22.242664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d606ad-1504-4abe-b790-170af4b1ab8e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:22.280643 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144e041-35fb-46a5-8079-4eeefa05cb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:22.281648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e041-35fb-46a5-8079-4eeefa05cb40 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.282113 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e041-35fb-46a5-8079-4eeefa05cb40 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.282638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e041-35fb-46a5-8079-4eeefa05cb40 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.283496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144e041-35fb-46a5-8079-4eeefa05cb40 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:22.283984 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144e041-35fb-46a5-8079-4eeefa05cb40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:22.285853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144e041-35fb-46a5-8079-4eeefa05cb40 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:22.291526 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 469/905] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:22.298537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144e041-35fb-46a5-8079-4eeefa05cb40 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:22.319884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:22.352326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b76cb-edf9-49ad-b070-29af6c5c3555 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:22.390195 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12b8b4-b632-41f8-b793-e7d7a67a20e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:22.390803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12b8b4-b632-41f8-b793-e7d7a67a20e0 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.391075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12b8b4-b632-41f8-b793-e7d7a67a20e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.391368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12b8b4-b632-41f8-b793-e7d7a67a20e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.392148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12b8b4-b632-41f8-b793-e7d7a67a20e0 None admin] RBAC: Adding query filter params (role.id=64b48b964be3471a89c5b9cf4fc9b990, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=ee318396a8a04174a509a5faa8b86aa1) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:22.392452 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12b8b4-b632-41f8-b793-e7d7a67a20e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:22.393877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12b8b4-b632-41f8-b793-e7d7a67a20e0 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:22.399406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 437/906] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:22 2022] GET /identity/v3/role_assignments?user.id=ee318396a8a04174a509a5faa8b86aa1&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=64b48b964be3471a89c5b9cf4fc9b990 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:22.404917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6c19d-f977-4314-9d88-2daa878889e6 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.405942 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6c19d-f977-4314-9d88-2daa878889e6 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.406410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6c19d-f977-4314-9d88-2daa878889e6 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.408464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 438/907] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:22.417435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9a5f-ec15-48f5-9ca2-5f1c228c6e16 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.418293 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9a5f-ec15-48f5-9ca2-5f1c228c6e16 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.418778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9a5f-ec15-48f5-9ca2-5f1c228c6e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.427013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144e041-35fb-46a5-8079-4eeefa05cb40 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:22.443873 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-d40a9a5f-ec15-48f5-9ca2-5f1c228c6e16 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:22.445724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-d40a9a5f-ec15-48f5-9ca2-5f1c228c6e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:22.484658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405683d-5975-4e26-b865-a9e9f09e9643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:22.485584 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405683d-5975-4e26-b865-a9e9f09e9643 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.485999 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405683d-5975-4e26-b865-a9e9f09e9643 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.486423 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405683d-5975-4e26-b865-a9e9f09e9643 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.487700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405683d-5975-4e26-b865-a9e9f09e9643 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:22.489342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405683d-5975-4e26-b865-a9e9f09e9643 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:22.490912 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405683d-5975-4e26-b865-a9e9f09e9643 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:22.498400 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 470/908] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:22.504003 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405683d-5975-4e26-b865-a9e9f09e9643 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:22.525712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:22.532915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 439/909] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 30 11:37:22.598292 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:22.620702 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405683d-5975-4e26-b865-a9e9f09e9643 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:22.655474 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eef49-a7a0-49bb-a63d-3c28fb332eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:22.656127 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eef49-a7a0-49bb-a63d-3c28fb332eb5 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.656398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eef49-a7a0-49bb-a63d-3c28fb332eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.656659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eef49-a7a0-49bb-a63d-3c28fb332eb5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.657399 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eef49-a7a0-49bb-a63d-3c28fb332eb5 None admin] RBAC: Adding query filter params (interface=public, service_id=dea9f45dd63849fdbb150bb2c9916365, region_id=RegionOne) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:22.657700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eef49-a7a0-49bb-a63d-3c28fb332eb5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:22.659058 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eef49-a7a0-49bb-a63d-3c28fb332eb5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:22.664723 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 471/910] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Aug 30 11:37:22 2022] GET /identity/v3/endpoints?service_id=dea9f45dd63849fdbb150bb2c9916365&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:22.670236 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a9a5f-ec15-48f5-9ca2-5f1c228c6e16 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:22.775003 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a9a5f-ec15-48f5-9ca2-5f1c228c6e16 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:22.797353 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eef49-a7a0-49bb-a63d-3c28fb332eb5 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:22.813519 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad569af-05fc-4369-acc0-48dc8de88ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:22.814131 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad569af-05fc-4369-acc0-48dc8de88ed7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.814403 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad569af-05fc-4369-acc0-48dc8de88ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.814669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad569af-05fc-4369-acc0-48dc8de88ed7 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.815315 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad569af-05fc-4369-acc0-48dc8de88ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:22.816729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad569af-05fc-4369-acc0-48dc8de88ed7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:22.822719 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 440/911] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Aug 30 11:37:22 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 11:37:22.823930 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0871b2-017c-4eb9-bc4b-a1d107287ec1 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.824207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0871b2-017c-4eb9-bc4b-a1d107287ec1 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.824475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0871b2-017c-4eb9-bc4b-a1d107287ec1 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.825919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 441/912] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:22.833083 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b704c-5246-484e-9c81-44c1555262ba None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.833389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b704c-5246-484e-9c81-44c1555262ba None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.833633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b704c-5246-484e-9c81-44c1555262ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.857469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-158b704c-5246-484e-9c81-44c1555262ba None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:22.859136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-158b704c-5246-484e-9c81-44c1555262ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:22.925740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eef49-a7a0-49bb-a63d-3c28fb332eb5 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:22.944627 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:22.958022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 442/913] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:22.972202 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd57ed0-5542-40bb-b849-3a1cf4d899f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:22.973254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd57ed0-5542-40bb-b849-3a1cf4d899f9 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:22.973754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd57ed0-5542-40bb-b849-3a1cf4d899f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:22.974228 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd57ed0-5542-40bb-b849-3a1cf4d899f9 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:22.975102 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd57ed0-5542-40bb-b849-3a1cf4d899f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:22.976833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd57ed0-5542-40bb-b849-3a1cf4d899f9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:22.996662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 472/914] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:37:22 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:37:23.227194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b704c-5246-484e-9c81-44c1555262ba None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:23.330102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b704c-5246-484e-9c81-44c1555262ba None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:23.372200 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc6ae8-deff-45ed-9f4a-751bb437f6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:23.372874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6ae8-deff-45ed-9f4a-751bb437f6a9 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:23.373153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6ae8-deff-45ed-9f4a-751bb437f6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:23.373423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6ae8-deff-45ed-9f4a-751bb437f6a9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:23.374004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc6ae8-deff-45ed-9f4a-751bb437f6a9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:23.375483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc6ae8-deff-45ed-9f4a-751bb437f6a9 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-10bc6ae8-deff-45ed-9f4a-751bb437f6a9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:23.386832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:23.387860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:23.388714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 443/915] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:37:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:23.394532 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd57ed0-5542-40bb-b849-3a1cf4d899f9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:23.397112 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:23.501177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd57ed0-5542-40bb-b849-3a1cf4d899f9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:23.535999 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54694f81-3bdb-4589-bb8b-56375c2c937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:23.536622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54694f81-3bdb-4589-bb8b-56375c2c937b None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:23.536897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54694f81-3bdb-4589-bb8b-56375c2c937b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:23.537163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54694f81-3bdb-4589-bb8b-56375c2c937b None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:23.537802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54694f81-3bdb-4589-bb8b-56375c2c937b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:23.538101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54694f81-3bdb-4589-bb8b-56375c2c937b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:23.539520 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54694f81-3bdb-4589-bb8b-56375c2c937b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:23.544839 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 473/916] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:37:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:23.550554 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc6ae8-deff-45ed-9f4a-751bb437f6a9 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:23.607034 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:23.658741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc6ae8-deff-45ed-9f4a-751bb437f6a9 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:23.696088 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c324297-45a3-4f98-b791-da6801632690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:23.696697 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c324297-45a3-4f98-b791-da6801632690 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:23.697013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c324297-45a3-4f98-b791-da6801632690 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:23.697283 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c324297-45a3-4f98-b791-da6801632690 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:23.697916 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c324297-45a3-4f98-b791-da6801632690 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:23.698214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c324297-45a3-4f98-b791-da6801632690 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:23.699620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c324297-45a3-4f98-b791-da6801632690 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:23.705268 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 444/917] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Aug 30 11:37:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:23.710920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54694f81-3bdb-4589-bb8b-56375c2c937b None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:23.780388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:23.816687 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54694f81-3bdb-4589-bb8b-56375c2c937b None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:23.851729 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c694051-f3e1-4464-a409-2a63b5a413d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:23.852701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694051-f3e1-4464-a409-2a63b5a413d4 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:23.853172 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694051-f3e1-4464-a409-2a63b5a413d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:23.853662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694051-f3e1-4464-a409-2a63b5a413d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:23.854527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c694051-f3e1-4464-a409-2a63b5a413d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:23.856182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c694051-f3e1-4464-a409-2a63b5a413d4 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:23.880339 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 474/918] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:37:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:24.396613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80520ff9-2708-4d81-a35b-58b4d2b0f80d None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:24.397165 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80520ff9-2708-4d81-a35b-58b4d2b0f80d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:24.397165 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80520ff9-2708-4d81-a35b-58b4d2b0f80d None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:24.398735 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 445/919] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:24.404279 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab453f0-6461-4879-a597-20e9a58b3b4d None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:24.404700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab453f0-6461-4879-a597-20e9a58b3b4d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:24.404878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab453f0-6461-4879-a597-20e9a58b3b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:24.430223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-fab453f0-6461-4879-a597-20e9a58b3b4d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:24.431499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-fab453f0-6461-4879-a597-20e9a58b3b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:24.513891 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 475/920] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:24.780712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80520ff9-2708-4d81-a35b-58b4d2b0f80d None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:24.845678 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e8a7e-f74a-4853-a3f2-a63c230fd07b None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:24.845975 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e8a7e-f74a-4853-a3f2-a63c230fd07b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:24.846410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e8a7e-f74a-4853-a3f2-a63c230fd07b None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:24.848167 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 476/921] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:24.852988 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611bfed-514f-4188-8224-f5aaed2c9047 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:24.853327 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611bfed-514f-4188-8224-f5aaed2c9047 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:24.853665 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611bfed-514f-4188-8224-f5aaed2c9047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:24.873919 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-c611bfed-514f-4188-8224-f5aaed2c9047 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:24.875254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c611bfed-514f-4188-8224-f5aaed2c9047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:24.887659 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80520ff9-2708-4d81-a35b-58b4d2b0f80d None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:24.909769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:24.927608 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:24.930346 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d057b-6d48-45e8-ad30-0eb66bf07df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:24.931050 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d057b-6d48-45e8-ad30-0eb66bf07df2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:24.931412 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d057b-6d48-45e8-ad30-0eb66bf07df2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:24.931718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d057b-6d48-45e8-ad30-0eb66bf07df2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:24.945000 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d057b-6d48-45e8-ad30-0eb66bf07df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-761d057b-6d48-45e8-ad30-0eb66bf07df2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:24.951493 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:24.952609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:24.954207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 446/922] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Aug 30 11:37:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:24.954983 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82c043-381b-40dd-9e0a-2371ceac9e38 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:24.954983 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82c043-381b-40dd-9e0a-2371ceac9e38 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:24.955562 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82c043-381b-40dd-9e0a-2371ceac9e38 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:24.960758 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 447/923] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:24.960758 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82c043-381b-40dd-9e0a-2371ceac9e38 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:24.960965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 477/924] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:24.964578 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea250e-b571-437f-aa45-807668cb2c22 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:24.964862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea250e-b571-437f-aa45-807668cb2c22 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:24.965129 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea250e-b571-437f-aa45-807668cb2c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:24.990356 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-fbea250e-b571-437f-aa45-807668cb2c22 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:24.991581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-fbea250e-b571-437f-aa45-807668cb2c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:25.062145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 478/925] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:25.075526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82c043-381b-40dd-9e0a-2371ceac9e38 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.109843 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa560729-5216-4d2b-a9a9-cb97a5e03e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.110450 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa560729-5216-4d2b-a9a9-cb97a5e03e74 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.110721 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa560729-5216-4d2b-a9a9-cb97a5e03e74 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.110985 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa560729-5216-4d2b-a9a9-cb97a5e03e74 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.111676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa560729-5216-4d2b-a9a9-cb97a5e03e74 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:25.111974 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa560729-5216-4d2b-a9a9-cb97a5e03e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.113381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa560729-5216-4d2b-a9a9-cb97a5e03e74 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.122167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 448/926] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Aug 30 11:37:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:25.127694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea250e-b571-437f-aa45-807668cb2c22 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.230565 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea250e-b571-437f-aa45-807668cb2c22 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.236668 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa560729-5216-4d2b-a9a9-cb97a5e03e74 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.240751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:25.277323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e746d-3dec-48bf-9821-f54b48791ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.277967 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e746d-3dec-48bf-9821-f54b48791ed6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.278242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e746d-3dec-48bf-9821-f54b48791ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.278512 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e746d-3dec-48bf-9821-f54b48791ed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.280171 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:25.290549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e746d-3dec-48bf-9821-f54b48791ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.292692 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e746d-3dec-48bf-9821-f54b48791ed6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-c09e746d-3dec-48bf-9821-f54b48791ed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:25.311262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:25.313367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:25.314613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:25.314613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:25.314613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:25.314613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 479/927] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:25.318963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e746d-3dec-48bf-9821-f54b48791ed6 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.353388 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa560729-5216-4d2b-a9a9-cb97a5e03e74 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.388106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62625037-13f1-45f3-a2dc-7e7f9fdefb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.388730 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62625037-13f1-45f3-a2dc-7e7f9fdefb6d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.389003 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62625037-13f1-45f3-a2dc-7e7f9fdefb6d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.389266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62625037-13f1-45f3-a2dc-7e7f9fdefb6d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.389844 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62625037-13f1-45f3-a2dc-7e7f9fdefb6d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.391367 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62625037-13f1-45f3-a2dc-7e7f9fdefb6d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-62625037-13f1-45f3-a2dc-7e7f9fdefb6d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:25.401220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:25.402584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:25.403895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 449/928] 10.222.0.16 () {56 vars in 1132 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:25.404726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62625037-13f1-45f3-a2dc-7e7f9fdefb6d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.429841 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e746d-3dec-48bf-9821-f54b48791ed6 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.467982 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc6ecd-09d8-4eef-bf35-7e6077169ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.468596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc6ecd-09d8-4eef-bf35-7e6077169ee4 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.468870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc6ecd-09d8-4eef-bf35-7e6077169ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.469136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc6ecd-09d8-4eef-bf35-7e6077169ee4 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.469771 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc6ecd-09d8-4eef-bf35-7e6077169ee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:25.470068 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc6ecd-09d8-4eef-bf35-7e6077169ee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.471482 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc6ecd-09d8-4eef-bf35-7e6077169ee4 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.489157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 480/929] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:25.490510 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc6ecd-09d8-4eef-bf35-7e6077169ee4 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.512997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:25.514953 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62625037-13f1-45f3-a2dc-7e7f9fdefb6d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.516531 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:25.551502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043d065-aa75-4c41-9230-8c7c646f5310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.552139 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043d065-aa75-4c41-9230-8c7c646f5310 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.552403 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043d065-aa75-4c41-9230-8c7c646f5310 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.552670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043d065-aa75-4c41-9230-8c7c646f5310 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.553255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043d065-aa75-4c41-9230-8c7c646f5310 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.554622 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043d065-aa75-4c41-9230-8c7c646f5310 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-0043d065-aa75-4c41-9230-8c7c646f5310 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:25.563945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:25.564924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:25.568702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 450/930] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:25.569497 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043d065-aa75-4c41-9230-8c7c646f5310 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.594345 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc6ecd-09d8-4eef-bf35-7e6077169ee4 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.628893 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dab087-beac-460d-b1a6-367a1e40c3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.629499 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dab087-beac-460d-b1a6-367a1e40c3ea None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.629769 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dab087-beac-460d-b1a6-367a1e40c3ea None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.630030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dab087-beac-460d-b1a6-367a1e40c3ea None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.630651 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dab087-beac-460d-b1a6-367a1e40c3ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:25.630943 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dab087-beac-460d-b1a6-367a1e40c3ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.632331 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dab087-beac-460d-b1a6-367a1e40c3ea None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.641467 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 481/931] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:25.641467 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dab087-beac-460d-b1a6-367a1e40c3ea None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.680081 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043d065-aa75-4c41-9230-8c7c646f5310 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.716621 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0822bc4-2310-429c-80b5-84c256767e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.717249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0822bc4-2310-429c-80b5-84c256767e2a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.717520 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0822bc4-2310-429c-80b5-84c256767e2a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.717780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0822bc4-2310-429c-80b5-84c256767e2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.722385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:25.726353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0822bc4-2310-429c-80b5-84c256767e2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.727852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0822bc4-2310-429c-80b5-84c256767e2a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.738526 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dab087-beac-460d-b1a6-367a1e40c3ea None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-e0822bc4-2310-429c-80b5-84c256767e2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:25.740647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:25.741705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 451/932] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:25.742667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0822bc4-2310-429c-80b5-84c256767e2a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.770038 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:25.781026 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7e7b9-f0de-4ed5-bf41-96ad0bf44253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.781636 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7e7b9-f0de-4ed5-bf41-96ad0bf44253 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.781906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7e7b9-f0de-4ed5-bf41-96ad0bf44253 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.782163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7e7b9-f0de-4ed5-bf41-96ad0bf44253 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.782871 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7e7b9-f0de-4ed5-bf41-96ad0bf44253 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:25.783196 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7e7b9-f0de-4ed5-bf41-96ad0bf44253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.784718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7e7b9-f0de-4ed5-bf41-96ad0bf44253 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.794555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 482/933] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:25.794555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7e7b9-f0de-4ed5-bf41-96ad0bf44253 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.857305 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0822bc4-2310-429c-80b5-84c256767e2a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.892434 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1e7f6-f0e6-4fa2-8a9e-cdf26b3d3cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.893034 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1e7f6-f0e6-4fa2-8a9e-cdf26b3d3cfa None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.893308 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1e7f6-f0e6-4fa2-8a9e-cdf26b3d3cfa None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.893557 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7e7b9-f0de-4ed5-bf41-96ad0bf44253 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:25.893656 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1e7f6-f0e6-4fa2-8a9e-cdf26b3d3cfa None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.894188 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1e7f6-f0e6-4fa2-8a9e-cdf26b3d3cfa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:25.894487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1e7f6-f0e6-4fa2-8a9e-cdf26b3d3cfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.895883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1e7f6-f0e6-4fa2-8a9e-cdf26b3d3cfa None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.900518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 452/934] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:25.900518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1e7f6-f0e6-4fa2-8a9e-cdf26b3d3cfa None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.930878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0ca91-7676-426e-8ec5-a8241aed0b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:25.931594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0ca91-7676-426e-8ec5-a8241aed0b02 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.931925 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0ca91-7676-426e-8ec5-a8241aed0b02 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.932178 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0ca91-7676-426e-8ec5-a8241aed0b02 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.932907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0ca91-7676-426e-8ec5-a8241aed0b02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:25.933200 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0ca91-7676-426e-8ec5-a8241aed0b02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:25.934654 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0ca91-7676-426e-8ec5-a8241aed0b02 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:25.951680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 483/935] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:25.953186 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa87790-9051-43b1-b93e-4ac3d2bfda9b None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:25.953954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa87790-9051-43b1-b93e-4ac3d2bfda9b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:25.954353 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa87790-9051-43b1-b93e-4ac3d2bfda9b None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:25.955760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 484/936] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:25.956571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa87790-9051-43b1-b93e-4ac3d2bfda9b None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:25.965393 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:25.992702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:26.004029 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1e7f6-f0e6-4fa2-8a9e-cdf26b3d3cfa None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.041591 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dbba3-673e-4519-9fa8-44bfcf88c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.042204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dbba3-673e-4519-9fa8-44bfcf88c89a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.042472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dbba3-673e-4519-9fa8-44bfcf88c89a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.042734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dbba3-673e-4519-9fa8-44bfcf88c89a None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.043390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dbba3-673e-4519-9fa8-44bfcf88c89a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:26.043686 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dbba3-673e-4519-9fa8-44bfcf88c89a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.045048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dbba3-673e-4519-9fa8-44bfcf88c89a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.049772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 453/937] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:26.050566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dbba3-673e-4519-9fa8-44bfcf88c89a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:26.063339 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa87790-9051-43b1-b93e-4ac3d2bfda9b None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.096521 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198ccbf-7d07-44d2-b32e-247c5c16be33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.097122 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198ccbf-7d07-44d2-b32e-247c5c16be33 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.097446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198ccbf-7d07-44d2-b32e-247c5c16be33 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.097708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198ccbf-7d07-44d2-b32e-247c5c16be33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.098363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198ccbf-7d07-44d2-b32e-247c5c16be33 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.099759 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198ccbf-7d07-44d2-b32e-247c5c16be33 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.120695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 485/938] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:37:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:26.121957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d423f38-9f49-4297-9971-087ca9f98fa9 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.122312 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d423f38-9f49-4297-9971-087ca9f98fa9 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.122614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d423f38-9f49-4297-9971-087ca9f98fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.149392 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-0d423f38-9f49-4297-9971-087ca9f98fa9 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:26.150960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-0d423f38-9f49-4297-9971-087ca9f98fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:26.157792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dbba3-673e-4519-9fa8-44bfcf88c89a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.200260 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34262ae-fc22-4377-8f52-acc0419ff9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.200905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34262ae-fc22-4377-8f52-acc0419ff9bc None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.201176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34262ae-fc22-4377-8f52-acc0419ff9bc None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.201433 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34262ae-fc22-4377-8f52-acc0419ff9bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.214431 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34262ae-fc22-4377-8f52-acc0419ff9bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.215938 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34262ae-fc22-4377-8f52-acc0419ff9bc None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.216050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:26.240631 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 486/939] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-b34262ae-fc22-4377-8f52-acc0419ff9bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:26.240853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:26.241728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:26.242618 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:26.242618 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:26.242618 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:26.242789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d423f38-9f49-4297-9971-087ca9f98fa9 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:26.242858 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 454/940] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:26.242858 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34262ae-fc22-4377-8f52-acc0419ff9bc None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:26.245914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:26.364208 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34262ae-fc22-4377-8f52-acc0419ff9bc None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.384725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d423f38-9f49-4297-9971-087ca9f98fa9 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.402868 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6841e6-8553-43c2-a9de-9c850b73d597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.403640 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6841e6-8553-43c2-a9de-9c850b73d597 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.403984 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6841e6-8553-43c2-a9de-9c850b73d597 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.404316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6841e6-8553-43c2-a9de-9c850b73d597 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.405139 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6841e6-8553-43c2-a9de-9c850b73d597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:26.405568 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6841e6-8553-43c2-a9de-9c850b73d597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.407016 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6841e6-8553-43c2-a9de-9c850b73d597 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.420169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 455/941] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:26.425783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6841e6-8553-43c2-a9de-9c850b73d597 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:26.432916 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656391-7297-44dd-b456-393c5d69ba45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.433536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656391-7297-44dd-b456-393c5d69ba45 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.433807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656391-7297-44dd-b456-393c5d69ba45 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.434071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656391-7297-44dd-b456-393c5d69ba45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.434768 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d656391-7297-44dd-b456-393c5d69ba45 None admin] RBAC: Adding query filter params (interface=public, service_id=b29c9d91a4764996924dfb6c64f9826f, region_id=RegionOne) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:26.435066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d656391-7297-44dd-b456-393c5d69ba45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.436486 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d656391-7297-44dd-b456-393c5d69ba45 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.448660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 487/942] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Aug 30 11:37:26 2022] GET /identity/v3/endpoints?service_id=b29c9d91a4764996924dfb6c64f9826f&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:26.450559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656391-7297-44dd-b456-393c5d69ba45 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:26.499077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:26.527920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:26.539898 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6841e6-8553-43c2-a9de-9c850b73d597 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.574598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656391-7297-44dd-b456-393c5d69ba45 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.575041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f0819-238f-4825-8dab-3001e87ec462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.575713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f0819-238f-4825-8dab-3001e87ec462 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.576124 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f0819-238f-4825-8dab-3001e87ec462 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.576415 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f0819-238f-4825-8dab-3001e87ec462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.585390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f0819-238f-4825-8dab-3001e87ec462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.587200 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f0819-238f-4825-8dab-3001e87ec462 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-fa9f0819-238f-4825-8dab-3001e87ec462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:26.598892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:26.599836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:26.600740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:26.600740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:26.600740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:26.600740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:26.600740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:26.600740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:26.600740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:26.600740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:26.605186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 456/943] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:26.605495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f0819-238f-4825-8dab-3001e87ec462 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:26.610612 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde82a-c35d-49ae-b04d-4373e68b47b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.611255 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde82a-c35d-49ae-b04d-4373e68b47b7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.611528 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde82a-c35d-49ae-b04d-4373e68b47b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.611791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde82a-c35d-49ae-b04d-4373e68b47b7 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.612390 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cde82a-c35d-49ae-b04d-4373e68b47b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.614206 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cde82a-c35d-49ae-b04d-4373e68b47b7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.621088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 488/944] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Aug 30 11:37:26 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 11:37:26.621879 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde82a-c35d-49ae-b04d-4373e68b47b7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:26.739889 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde82a-c35d-49ae-b04d-4373e68b47b7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.740401 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f0819-238f-4825-8dab-3001e87ec462 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.762523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:26.765427 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:26.784949 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.785568 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.785843 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.786146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.786756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.787387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665324-22ce-4b39-b6f8-d1ec0af79993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.788004 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665324-22ce-4b39-b6f8-d1ec0af79993 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.788227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.788323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665324-22ce-4b39-b6f8-d1ec0af79993 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.788525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665324-22ce-4b39-b6f8-d1ec0af79993 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.789216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09665324-22ce-4b39-b6f8-d1ec0af79993 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:26.789514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09665324-22ce-4b39-b6f8-d1ec0af79993 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.790825 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: WARNING keystone.server.flask.common [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 11:37:26.790922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09665324-22ce-4b39-b6f8-d1ec0af79993 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.803544 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 489/945] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:26.809535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665324-22ce-4b39-b6f8-d1ec0af79993 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:26.814573 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.sql.core [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 11:37:26.814573 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 11:37:26.814573 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [parameters: {'id': '0d7c6688fa9942b2a49de17bd9b2209b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 11:37:26.814573 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87979) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0d7c6688fa9942b2a49de17bd9b2209b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0d7c6688fa9942b2a49de17bd9b2209b. Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application conn.query(q) Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 11:37:26.834119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application result.read() Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 11:37:26.834998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application next(self.gen) Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application yield resource Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application next(self.gen) Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise self.value Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application session.commit() Aug 30 11:37:26.836013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.session.flush() Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application compat.raise_( Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 11:37:26.836921 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 11:37:26.837826 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application conn.query(q) Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application result.read() Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 11:37:26.838602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [parameters: {'id': '0d7c6688fa9942b2a49de17bd9b2209b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:26.839676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:26.840710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:26.842250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:26.842250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 11:37:26.842250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 11:37:26.842250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:26.842250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:26.842250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 11:37:26.842250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 11:37:26.842250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0d7c6688fa9942b2a49de17bd9b2209b. Aug 30 11:37:26.842250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:26.845193 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 457/946] 10.222.0.16 () {60 vars in 1140 bytes} [Tue Aug 30 11:37:26 2022] POST /identity/v3/projects => generated 262 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:26.851052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:26.924449 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665324-22ce-4b39-b6f8-d1ec0af79993 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.959767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef254b19-becd-4878-bcd7-ad0ff94892d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:26.960518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef254b19-becd-4878-bcd7-ad0ff94892d5 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:26.961129 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef254b19-becd-4878-bcd7-ad0ff94892d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:26.961129 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef254b19-becd-4878-bcd7-ad0ff94892d5 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/ee318396a8a04174a509a5faa8b86aa1/roles/64b48b964be3471a89c5b9cf4fc9b990` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:26.963461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef254b19-becd-4878-bcd7-ad0ff94892d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=ee318396a8a04174a509a5faa8b86aa1, role_id=64b48b964be3471a89c5b9cf4fc9b990)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:26.964975 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef254b19-becd-4878-bcd7-ad0ff94892d5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:26.982739 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211bf28-13e7-4481-9019-157b3e934589 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:26.987214 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 490/947] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:37:26 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/ee318396a8a04174a509a5faa8b86aa1/roles/64b48b964be3471a89c5b9cf4fc9b990 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 11:37:27.030115 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455f606-6131-4e2a-b95e-a94797511daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:27.031172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455f606-6131-4e2a-b95e-a94797511daa None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:27.031668 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455f606-6131-4e2a-b95e-a94797511daa None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:27.032216 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455f606-6131-4e2a-b95e-a94797511daa None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:27.040961 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455f606-6131-4e2a-b95e-a94797511daa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:27.043210 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455f606-6131-4e2a-b95e-a94797511daa None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-1455f606-6131-4e2a-b95e-a94797511daa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:27.057227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:27.058238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:27.059155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:27.059155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:27.059155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:27.059155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:27.059155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:27.059155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:27.059155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:27.059155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:27.061752 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 458/948] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:27.068329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef254b19-becd-4878-bcd7-ad0ff94892d5 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:27.119475 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:27.181614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef254b19-becd-4878-bcd7-ad0ff94892d5 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:27.216289 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee4591-2555-4aca-92ff-d2c265ea277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:27.217289 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee4591-2555-4aca-92ff-d2c265ea277e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:27.217771 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee4591-2555-4aca-92ff-d2c265ea277e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:27.218240 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee4591-2555-4aca-92ff-d2c265ea277e None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:27.219104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee4591-2555-4aca-92ff-d2c265ea277e None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:27.219662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee4591-2555-4aca-92ff-d2c265ea277e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:27.221363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee4591-2555-4aca-92ff-d2c265ea277e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:27.239555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 491/949] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:28.109958 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee5057-059b-4ca7-9b5f-1e96e952f47c None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:28.110562 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee5057-059b-4ca7-9b5f-1e96e952f47c None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:28.110562 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee5057-059b-4ca7-9b5f-1e96e952f47c None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:28.115138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 459/950] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:28.120449 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07647c-b3e9-4d18-869d-12f2dd6d0c98 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:28.120894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07647c-b3e9-4d18-869d-12f2dd6d0c98 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:28.120980 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07647c-b3e9-4d18-869d-12f2dd6d0c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:28.139452 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-5c07647c-b3e9-4d18-869d-12f2dd6d0c98 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:28.140689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-5c07647c-b3e9-4d18-869d-12f2dd6d0c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:28.207928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 492/951] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:28.471625 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee5057-059b-4ca7-9b5f-1e96e952f47c None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:28.514732 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:28.577914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee5057-059b-4ca7-9b5f-1e96e952f47c None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:28.606965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c345ab-35b5-4cd4-91f9-21c59141c76e None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:28.607312 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c345ab-35b5-4cd4-91f9-21c59141c76e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:28.607553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c345ab-35b5-4cd4-91f9-21c59141c76e None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:28.609131 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 493/952] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:28.614503 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318ecf7-6e73-4987-951f-e4fda5a8a1ad None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:28.614781 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318ecf7-6e73-4987-951f-e4fda5a8a1ad None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:28.615052 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318ecf7-6e73-4987-951f-e4fda5a8a1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:28.616557 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1d6e4-099e-4ad1-b9c6-3a7dee47de77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:28.617437 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1d6e4-099e-4ad1-b9c6-3a7dee47de77 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:28.617920 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1d6e4-099e-4ad1-b9c6-3a7dee47de77 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:28.618391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1d6e4-099e-4ad1-b9c6-3a7dee47de77 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:28.619247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1d6e4-099e-4ad1-b9c6-3a7dee47de77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:28.620820 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1d6e4-099e-4ad1-b9c6-3a7dee47de77 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-1fc1d6e4-099e-4ad1-b9c6-3a7dee47de77 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:28.630777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:28.632161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 11:37:28.633244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:28.634554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-a318ecf7-6e73-4987-951f-e4fda5a8a1ad None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:28.635795 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-a318ecf7-6e73-4987-951f-e4fda5a8a1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:28.639328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 460/953] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:28.642644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1d6e4-099e-4ad1-b9c6-3a7dee47de77 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:28.663019 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:28.723860 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 494/954] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:28.760829 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1d6e4-099e-4ad1-b9c6-3a7dee47de77 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:28.762706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:28.797139 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b54a21-922c-4d89-a854-b118be3e9b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:28.798128 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b54a21-922c-4d89-a854-b118be3e9b9e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:28.798620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b54a21-922c-4d89-a854-b118be3e9b9e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:28.799092 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b54a21-922c-4d89-a854-b118be3e9b9e None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:28.799970 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b54a21-922c-4d89-a854-b118be3e9b9e None admin] RBAC: Adding query filter params (name=compute) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:28.800525 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b54a21-922c-4d89-a854-b118be3e9b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:28.802089 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b54a21-922c-4d89-a854-b118be3e9b9e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:28.811235 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 461/955] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:28.813697 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318ecf7-6e73-4987-951f-e4fda5a8a1ad None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:28.915949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318ecf7-6e73-4987-951f-e4fda5a8a1ad None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:28.952922 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cefab-8af8-4925-8df9-6cacf6db11e2 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:28.953463 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6b470-d032-4796-b89f-23129bbfff41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:28.954147 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cefab-8af8-4925-8df9-6cacf6db11e2 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:28.954427 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6b470-d032-4796-b89f-23129bbfff41 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:28.954699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6b470-d032-4796-b89f-23129bbfff41 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:28.954962 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6b470-d032-4796-b89f-23129bbfff41 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:28.955258 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cefab-8af8-4925-8df9-6cacf6db11e2 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:28.955707 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6b470-d032-4796-b89f-23129bbfff41 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:28.955915 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6b470-d032-4796-b89f-23129bbfff41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:28.957522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6b470-d032-4796-b89f-23129bbfff41 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:28.961129 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 462/956] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:28.962756 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 495/957] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:28.963928 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196449-ce48-45c6-b684-734ab4f4b09a None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:28.964506 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196449-ce48-45c6-b684-734ab4f4b09a None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:28.965066 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196449-ce48-45c6-b684-734ab4f4b09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:28.986132 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-f4196449-ce48-45c6-b684-734ab4f4b09a None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:28.988232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-f4196449-ce48-45c6-b684-734ab4f4b09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:29.003470 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6b470-d032-4796-b89f-23129bbfff41 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:29.031197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:29.087179 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 463/958] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:29.121149 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6b470-d032-4796-b89f-23129bbfff41 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:29.158074 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fbb91-3651-44c5-9563-2c1bff917213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:29.158977 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fbb91-3651-44c5-9563-2c1bff917213 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.159431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fbb91-3651-44c5-9563-2c1bff917213 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.159947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fbb91-3651-44c5-9563-2c1bff917213 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.160686 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fbb91-3651-44c5-9563-2c1bff917213 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:29.162206 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fbb91-3651-44c5-9563-2c1bff917213 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-982fbb91-3651-44c5-9563-2c1bff917213 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:29.171397 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:29.172689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:29.173658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:29.174687 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 496/959] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:29.180852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196449-ce48-45c6-b684-734ab4f4b09a None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:29.202890 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1415ea-f3e6-4615-8a1e-3f6843568dc6 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.203205 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1415ea-f3e6-4615-8a1e-3f6843568dc6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.203788 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1415ea-f3e6-4615-8a1e-3f6843568dc6 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.205245 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 497/960] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:29.211688 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d555c2d-6d29-4834-9f7f-dc3592182648 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.211972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d555c2d-6d29-4834-9f7f-dc3592182648 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.212244 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d555c2d-6d29-4834-9f7f-dc3592182648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.230986 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-1d555c2d-6d29-4834-9f7f-dc3592182648 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:29.232328 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-1d555c2d-6d29-4834-9f7f-dc3592182648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:29.285913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:29.293090 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196449-ce48-45c6-b684-734ab4f4b09a None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:29.303343 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 498/961] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:29.328206 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c096e9-bfda-4a0b-b14d-409087565d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:29.329520 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c096e9-bfda-4a0b-b14d-409087565d40 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.330054 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c096e9-bfda-4a0b-b14d-409087565d40 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.330523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c096e9-bfda-4a0b-b14d-409087565d40 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.331370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c096e9-bfda-4a0b-b14d-409087565d40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:29.331916 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c096e9-bfda-4a0b-b14d-409087565d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:29.333542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c096e9-bfda-4a0b-b14d-409087565d40 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:29.339061 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 464/962] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:29.344501 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d555c2d-6d29-4834-9f7f-dc3592182648 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:29.352196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c096e9-bfda-4a0b-b14d-409087565d40 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:29.367445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:29.470801 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d555c2d-6d29-4834-9f7f-dc3592182648 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:29.479239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c096e9-bfda-4a0b-b14d-409087565d40 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:29.515223 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4435bbb-11a6-477b-9cc8-4e095b5d4048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:29.516197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4435bbb-11a6-477b-9cc8-4e095b5d4048 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.516627 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4435bbb-11a6-477b-9cc8-4e095b5d4048 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.516787 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031a2c5-4b99-4090-8d18-6c79e5616995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:29.517198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4435bbb-11a6-477b-9cc8-4e095b5d4048 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.517591 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031a2c5-4b99-4090-8d18-6c79e5616995 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.517957 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031a2c5-4b99-4090-8d18-6c79e5616995 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.518267 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4435bbb-11a6-477b-9cc8-4e095b5d4048 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:29.518374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031a2c5-4b99-4090-8d18-6c79e5616995 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.518794 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4435bbb-11a6-477b-9cc8-4e095b5d4048 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:29.520724 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4435bbb-11a6-477b-9cc8-4e095b5d4048 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:29.527128 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 499/963] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:29.533969 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031a2c5-4b99-4090-8d18-6c79e5616995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:29.535716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4435bbb-11a6-477b-9cc8-4e095b5d4048 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-1031a2c5-4b99-4090-8d18-6c79e5616995 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:29.537187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:29.538358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:29.539488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:29.543338 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 465/964] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Aug 30 11:37:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:29.546892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031a2c5-4b99-4090-8d18-6c79e5616995 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:29.569395 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:29.634200 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:29.652553 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031a2c5-4b99-4090-8d18-6c79e5616995 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:29.654514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4435bbb-11a6-477b-9cc8-4e095b5d4048 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:29.688738 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2e2de-e97f-401d-a260-39148b7f7ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:29.689320 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2e2de-e97f-401d-a260-39148b7f7ff4 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.689596 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2e2de-e97f-401d-a260-39148b7f7ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.689862 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2e2de-e97f-401d-a260-39148b7f7ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.690516 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2e2de-e97f-401d-a260-39148b7f7ff4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:29.690832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2e2de-e97f-401d-a260-39148b7f7ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:29.691964 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0766d92-51e9-457c-b0d8-a144f864f60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:29.692264 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2e2de-e97f-401d-a260-39148b7f7ff4 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:29.692574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0766d92-51e9-457c-b0d8-a144f864f60f None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.692855 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0766d92-51e9-457c-b0d8-a144f864f60f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.693121 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0766d92-51e9-457c-b0d8-a144f864f60f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.693784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0766d92-51e9-457c-b0d8-a144f864f60f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:29.695170 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0766d92-51e9-457c-b0d8-a144f864f60f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:29.705504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 466/965] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Aug 30 11:37:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:29.706603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2e2de-e97f-401d-a260-39148b7f7ff4 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:29.715841 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 500/966] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:37:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:29.716683 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0766d92-51e9-457c-b0d8-a144f864f60f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:29.816663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:29.842763 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2e2de-e97f-401d-a260-39148b7f7ff4 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:29.864010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0766d92-51e9-457c-b0d8-a144f864f60f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:29.892812 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:29.893431 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.893708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.894003 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.894643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:29.896161 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:29.906914 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be777d-0a01-4a6e-b64d-0110f40be8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:29.908026 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be777d-0a01-4a6e-b64d-0110f40be8a8 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:29.908026 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be777d-0a01-4a6e-b64d-0110f40be8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:29.908510 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be777d-0a01-4a6e-b64d-0110f40be8a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:29.912616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:29.917163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.sql.core [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 11:37:29.917163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 11:37:29.917163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [parameters: {'user_id': '665a56d7cc9f4acab65090d778356ef5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 11:37:29.917163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87979) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 11:37:29.924069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be777d-0a01-4a6e-b64d-0110f40be8a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:29.925568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be777d-0a01-4a6e-b64d-0110f40be8a8 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application conn.query(q) Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application result.read() Aug 30 11:37:29.925679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 11:37:29.926586 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application next(self.gen) Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application yield resource Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application next(self.gen) Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise self.value Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application session.commit() Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 11:37:29.927633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.session.flush() Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application compat.raise_( Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 11:37:29.928571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 11:37:29.929526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application conn.query(q) Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application result.read() Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 11:37:29.930435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [parameters: {'user_id': '665a56d7cc9f4acab65090d778356ef5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:29.933336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 11:37:29.935523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 11:37:29.940779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 11:37:29.940779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 11:37:29.940779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 11:37:29.940779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 11:37:29.940779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 11:37:29.940779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:29.940779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 467/967] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Aug 30 11:37:29 2022] POST /identity/v3/users => generated 160 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:29.940779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-c4be777d-0a01-4a6e-b64d-0110f40be8a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:29.949773 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:29.951737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:29.952847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:29.952847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:29.952847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:29.952847 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 501/968] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:29.957927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be777d-0a01-4a6e-b64d-0110f40be8a8 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:30.052977 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700c7df-06d7-440b-8402-f53296ce1b04 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:30.082069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be777d-0a01-4a6e-b64d-0110f40be8a8 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:30.101596 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610f7b9-8b12-45d8-ab9d-9cf08a483cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:30.102644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610f7b9-8b12-45d8-ab9d-9cf08a483cb9 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:30.103016 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610f7b9-8b12-45d8-ab9d-9cf08a483cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:30.103569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610f7b9-8b12-45d8-ab9d-9cf08a483cb9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:30.113373 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610f7b9-8b12-45d8-ab9d-9cf08a483cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:30.115857 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610f7b9-8b12-45d8-ab9d-9cf08a483cb9 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:30.122902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801380f-bdc8-48d5-83ac-80801210141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:30.123589 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801380f-bdc8-48d5-83ac-80801210141e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:30.123863 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801380f-bdc8-48d5-83ac-80801210141e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:30.124133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801380f-bdc8-48d5-83ac-80801210141e None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:30.125070 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801380f-bdc8-48d5-83ac-80801210141e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:30.125372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801380f-bdc8-48d5-83ac-80801210141e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:30.126747 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801380f-bdc8-48d5-83ac-80801210141e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-2610f7b9-8b12-45d8-ab9d-9cf08a483cb9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:30.129783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:30.131140 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 11:37:30.132741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:30.135228 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 468/969] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:37:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:30.136729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610f7b9-8b12-45d8-ab9d-9cf08a483cb9 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:30.141244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:30.150144 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 502/970] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:30.151699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801380f-bdc8-48d5-83ac-80801210141e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:30.216614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:30.248541 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610f7b9-8b12-45d8-ab9d-9cf08a483cb9 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:30.285531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801380f-bdc8-48d5-83ac-80801210141e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:30.291700 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe460e3-e594-43df-b160-ea37d4a1cef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:30.292592 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe460e3-e594-43df-b160-ea37d4a1cef7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:30.293566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe460e3-e594-43df-b160-ea37d4a1cef7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:30.294037 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe460e3-e594-43df-b160-ea37d4a1cef7 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:30.295330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe460e3-e594-43df-b160-ea37d4a1cef7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:30.295651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe460e3-e594-43df-b160-ea37d4a1cef7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:30.297085 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe460e3-e594-43df-b160-ea37d4a1cef7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:30.324407 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e15f-e2c2-4a64-aacf-7ed51388b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:30.325036 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e15f-e2c2-4a64-aacf-7ed51388b93f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:30.325304 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e15f-e2c2-4a64-aacf-7ed51388b93f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:30.325567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e15f-e2c2-4a64-aacf-7ed51388b93f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:30.333451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512e15f-e2c2-4a64-aacf-7ed51388b93f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:30.334851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512e15f-e2c2-4a64-aacf-7ed51388b93f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:30.346426 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 469/971] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Aug 30 11:37:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-5512e15f-e2c2-4a64-aacf-7ed51388b93f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:30.351575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:30.352566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:30.353987 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 503/972] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:37:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:30.361039 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe460e3-e594-43df-b160-ea37d4a1cef7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:30.432647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:30.476282 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe460e3-e594-43df-b160-ea37d4a1cef7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:30.513707 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd57776-1edd-4b5b-8dba-fceb01a75dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:30.514323 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd57776-1edd-4b5b-8dba-fceb01a75dbd None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:30.514599 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd57776-1edd-4b5b-8dba-fceb01a75dbd None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:30.514866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd57776-1edd-4b5b-8dba-fceb01a75dbd None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:30.515637 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd57776-1edd-4b5b-8dba-fceb01a75dbd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:30.515939 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd57776-1edd-4b5b-8dba-fceb01a75dbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:30.517372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd57776-1edd-4b5b-8dba-fceb01a75dbd None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:30.537767 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 470/973] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Aug 30 11:37:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:30.543800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e15f-e2c2-4a64-aacf-7ed51388b93f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:30.652219 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e15f-e2c2-4a64-aacf-7ed51388b93f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:30.659625 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:30.687081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d09bb-43cb-4645-9520-48647ce8bd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:30.687717 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d09bb-43cb-4645-9520-48647ce8bd7c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:30.687984 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d09bb-43cb-4645-9520-48647ce8bd7c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:30.688217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d09bb-43cb-4645-9520-48647ce8bd7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:30.761085 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d09bb-43cb-4645-9520-48647ce8bd7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:30.762573 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d09bb-43cb-4645-9520-48647ce8bd7c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:30.870013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 504/974] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:30 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:37:30.871925 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd57776-1edd-4b5b-8dba-fceb01a75dbd None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:30.932149 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128c8d-3a9e-45cf-81f8-8fb9a5cd38ca None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:30.932374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128c8d-3a9e-45cf-81f8-8fb9a5cd38ca None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:30.932618 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128c8d-3a9e-45cf-81f8-8fb9a5cd38ca None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:30.933977 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 505/975] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:30.938821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232f33b-01d5-4fdd-a774-f65c7496cb3d None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:30.939100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232f33b-01d5-4fdd-a774-f65c7496cb3d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:30.939392 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232f33b-01d5-4fdd-a774-f65c7496cb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:30.952385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:30.960330 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-3232f33b-01d5-4fdd-a774-f65c7496cb3d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:30.961529 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-3232f33b-01d5-4fdd-a774-f65c7496cb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:30.982990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd57776-1edd-4b5b-8dba-fceb01a75dbd None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:31.006743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:31.022845 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184a7bf-c0c9-43a9-9425-cff11270e1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:31.023894 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184a7bf-c0c9-43a9-9425-cff11270e1a8 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:31.024298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184a7bf-c0c9-43a9-9425-cff11270e1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:31.024569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184a7bf-c0c9-43a9-9425-cff11270e1a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:31.033572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 506/976] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:31.040126 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184a7bf-c0c9-43a9-9425-cff11270e1a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:31.041674 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184a7bf-c0c9-43a9-9425-cff11270e1a8 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-3184a7bf-c0c9-43a9-9425-cff11270e1a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:31.057130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:31.058134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:31.063597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:31.063597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:31.063597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:31.063597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 471/977] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:31.065410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232f33b-01d5-4fdd-a774-f65c7496cb3d None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:31.167800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232f33b-01d5-4fdd-a774-f65c7496cb3d None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:31.201346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841db36f-04e6-4513-ae47-660f7086f7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:31.201879 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841db36f-04e6-4513-ae47-660f7086f7ab None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:31.202148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841db36f-04e6-4513-ae47-660f7086f7ab None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:31.202412 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841db36f-04e6-4513-ae47-660f7086f7ab None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:31.203053 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841db36f-04e6-4513-ae47-660f7086f7ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:31.203387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841db36f-04e6-4513-ae47-660f7086f7ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:31.205137 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841db36f-04e6-4513-ae47-660f7086f7ab None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:31.207679 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:31.218530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 507/978] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:31.224270 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184a7bf-c0c9-43a9-9425-cff11270e1a8 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:31.291506 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841db36f-04e6-4513-ae47-660f7086f7ab None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:31.327011 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184a7bf-c0c9-43a9-9425-cff11270e1a8 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:31.360551 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0a7d9-09c8-45d6-9b75-70d5a48cf4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:31.361179 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0a7d9-09c8-45d6-9b75-70d5a48cf4b5 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:31.361459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0a7d9-09c8-45d6-9b75-70d5a48cf4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:31.361751 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0a7d9-09c8-45d6-9b75-70d5a48cf4b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:31.368471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0a7d9-09c8-45d6-9b75-70d5a48cf4b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:31.369920 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0a7d9-09c8-45d6-9b75-70d5a48cf4b5 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-46d0a7d9-09c8-45d6-9b75-70d5a48cf4b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:31.381471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:31.382384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:31.383416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:31.383416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:31.383416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:31.383416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:31.383416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:31.383416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:31.383416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:31.383416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:31.383416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 472/979] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:31.388383 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0a7d9-09c8-45d6-9b75-70d5a48cf4b5 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:31.395629 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841db36f-04e6-4513-ae47-660f7086f7ab None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:31.426988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:31.440285 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68604551-3b4e-4e87-b80f-708cbe1f9dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:31.441304 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68604551-3b4e-4e87-b80f-708cbe1f9dc6 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:31.441586 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68604551-3b4e-4e87-b80f-708cbe1f9dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:31.441852 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68604551-3b4e-4e87-b80f-708cbe1f9dc6 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:31.442514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68604551-3b4e-4e87-b80f-708cbe1f9dc6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:31.443922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68604551-3b4e-4e87-b80f-708cbe1f9dc6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:31.471442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 508/980] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:37:31 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:37:31.506481 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0a7d9-09c8-45d6-9b75-70d5a48cf4b5 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:31.555776 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddde01-e8cb-49d1-abcc-c34df21e2341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:31.556393 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddde01-e8cb-49d1-abcc-c34df21e2341 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:31.556666 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddde01-e8cb-49d1-abcc-c34df21e2341 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:31.556930 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddde01-e8cb-49d1-abcc-c34df21e2341 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:31.557615 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ddde01-e8cb-49d1-abcc-c34df21e2341 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:31.557913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ddde01-e8cb-49d1-abcc-c34df21e2341 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:31.559386 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ddde01-e8cb-49d1-abcc-c34df21e2341 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:31.577601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 473/981] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:31.579909 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68604551-3b4e-4e87-b80f-708cbe1f9dc6 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:31.641400 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:31.685207 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68604551-3b4e-4e87-b80f-708cbe1f9dc6 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:31.697551 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458ade0-459d-4e3f-8a3a-f70a4cfa1f91 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:31.697837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458ade0-459d-4e3f-8a3a-f70a4cfa1f91 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:31.698107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458ade0-459d-4e3f-8a3a-f70a4cfa1f91 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:31.699652 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 474/982] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:31.704627 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d81af-797e-4599-bd96-2c240ca6de15 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:31.704907 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d81af-797e-4599-bd96-2c240ca6de15 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:31.705180 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d81af-797e-4599-bd96-2c240ca6de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:31.720153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd15771-1780-464b-8ce9-cf0b94dc68ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:31.720764 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd15771-1780-464b-8ce9-cf0b94dc68ae None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:31.721034 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd15771-1780-464b-8ce9-cf0b94dc68ae None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:31.721293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd15771-1780-464b-8ce9-cf0b94dc68ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:31.722095 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd15771-1780-464b-8ce9-cf0b94dc68ae None admin] RBAC: Adding query filter params (role.id=64b48b964be3471a89c5b9cf4fc9b990, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=ee318396a8a04174a509a5faa8b86aa1) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:31.722337 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd15771-1780-464b-8ce9-cf0b94dc68ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:31.723798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd15771-1780-464b-8ce9-cf0b94dc68ae None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:31.723900 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-c21d81af-797e-4599-bd96-2c240ca6de15 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:31.725390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c21d81af-797e-4599-bd96-2c240ca6de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:31.730945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 509/983] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:31 2022] GET /identity/v3/role_assignments?user.id=ee318396a8a04174a509a5faa8b86aa1&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=64b48b964be3471a89c5b9cf4fc9b990 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:31.801487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 475/984] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:32.064982 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd15771-1780-464b-8ce9-cf0b94dc68ae None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:32.172227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd15771-1780-464b-8ce9-cf0b94dc68ae None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:32.192898 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:32.208055 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fc65a-fac6-4839-8b2f-1ad4886501e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:32.208694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fc65a-fac6-4839-8b2f-1ad4886501e5 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:32.208969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fc65a-fac6-4839-8b2f-1ad4886501e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:32.209234 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fc65a-fac6-4839-8b2f-1ad4886501e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:32.209815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4fc65a-fac6-4839-8b2f-1ad4886501e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:32.211216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4fc65a-fac6-4839-8b2f-1ad4886501e5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-6c4fc65a-fac6-4839-8b2f-1ad4886501e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:32.220031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:32.221013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:32.221877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 510/985] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:32.229076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d81af-797e-4599-bd96-2c240ca6de15 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:32.248905 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:32.332608 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d81af-797e-4599-bd96-2c240ca6de15 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:32.367128 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11741dd9-00f2-47ed-8fec-d60740559cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:32.367754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11741dd9-00f2-47ed-8fec-d60740559cf2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:32.368029 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11741dd9-00f2-47ed-8fec-d60740559cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:32.368301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11741dd9-00f2-47ed-8fec-d60740559cf2 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:32.368929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11741dd9-00f2-47ed-8fec-d60740559cf2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:32.369227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11741dd9-00f2-47ed-8fec-d60740559cf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:32.370597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11741dd9-00f2-47ed-8fec-d60740559cf2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:32.375752 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 476/986] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:32.381341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fc65a-fac6-4839-8b2f-1ad4886501e5 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:32.498047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fc65a-fac6-4839-8b2f-1ad4886501e5 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:32.533022 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d08ad-ddb3-4b07-9d44-68af72a8b973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:32.533624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d08ad-ddb3-4b07-9d44-68af72a8b973 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:32.533889 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d08ad-ddb3-4b07-9d44-68af72a8b973 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:32.534154 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d08ad-ddb3-4b07-9d44-68af72a8b973 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:32.534797 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d08ad-ddb3-4b07-9d44-68af72a8b973 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:32.535091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d08ad-ddb3-4b07-9d44-68af72a8b973 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:32.536484 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d08ad-ddb3-4b07-9d44-68af72a8b973 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:32.542062 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 511/987] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:32.548119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11741dd9-00f2-47ed-8fec-d60740559cf2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:32.631962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:32.652639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11741dd9-00f2-47ed-8fec-d60740559cf2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:32.687330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de863bc1-4a22-49cc-b698-a3a231ea47e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:32.687937 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de863bc1-4a22-49cc-b698-a3a231ea47e0 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:32.688212 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de863bc1-4a22-49cc-b698-a3a231ea47e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:32.688482 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de863bc1-4a22-49cc-b698-a3a231ea47e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:32.689191 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de863bc1-4a22-49cc-b698-a3a231ea47e0 None admin] RBAC: Adding query filter params (interface=admin, service_id=b29c9d91a4764996924dfb6c64f9826f, region_id=RegionOne) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:32.689499 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de863bc1-4a22-49cc-b698-a3a231ea47e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:32.690927 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de863bc1-4a22-49cc-b698-a3a231ea47e0 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:32.696269 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 477/988] 10.222.0.16 () {56 vars in 1249 bytes} [Tue Aug 30 11:37:32 2022] GET /identity/v3/endpoints?service_id=b29c9d91a4764996924dfb6c64f9826f&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:32.701819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d08ad-ddb3-4b07-9d44-68af72a8b973 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:32.752504 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:32.807448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d08ad-ddb3-4b07-9d44-68af72a8b973 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:32.844136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97da718-816f-4bfe-b56d-0048a02f85c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:32.844743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da718-816f-4bfe-b56d-0048a02f85c6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:32.845012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da718-816f-4bfe-b56d-0048a02f85c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:32.845273 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da718-816f-4bfe-b56d-0048a02f85c6 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:32.845865 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97da718-816f-4bfe-b56d-0048a02f85c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:32.847335 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97da718-816f-4bfe-b56d-0048a02f85c6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:32.853888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 512/989] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Aug 30 11:37:32 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 11:37:33.380966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfffb8-0525-40cd-b365-5d46e8b5c46d None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:33.381540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfffb8-0525-40cd-b365-5d46e8b5c46d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:33.381540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfffb8-0525-40cd-b365-5d46e8b5c46d None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:33.383016 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 478/990] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:33.388315 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a469739-6bf6-4a40-bf64-2f89920f453b None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:33.388745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a469739-6bf6-4a40-bf64-2f89920f453b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:33.388823 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a469739-6bf6-4a40-bf64-2f89920f453b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:33.406993 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-5a469739-6bf6-4a40-bf64-2f89920f453b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:33.410906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-5a469739-6bf6-4a40-bf64-2f89920f453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:33.478888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 513/991] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:33.623415 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9d027-7945-4486-ac7c-5d542b664b20 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:33.623651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9d027-7945-4486-ac7c-5d542b664b20 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:33.623912 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9d027-7945-4486-ac7c-5d542b664b20 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:33.625451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 479/992] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:33.630493 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca7db-b2ea-49ef-b200-6fb4660f6942 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:33.630738 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca7db-b2ea-49ef-b200-6fb4660f6942 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:33.631100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca7db-b2ea-49ef-b200-6fb4660f6942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:33.649191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-16fca7db-b2ea-49ef-b200-6fb4660f6942 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:33.650346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-16fca7db-b2ea-49ef-b200-6fb4660f6942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:33.685393 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:33.718452 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 514/993] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:33.744754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9d027-7945-4486-ac7c-5d542b664b20 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:33.850661 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9d027-7945-4486-ac7c-5d542b664b20 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:33.884537 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1beb51-1726-44a9-999a-db56447c87a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:33.885092 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1beb51-1726-44a9-999a-db56447c87a4 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:33.885367 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1beb51-1726-44a9-999a-db56447c87a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:33.885636 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1beb51-1726-44a9-999a-db56447c87a4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:33.886215 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1beb51-1726-44a9-999a-db56447c87a4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:33.887663 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1beb51-1726-44a9-999a-db56447c87a4 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-3a1beb51-1726-44a9-999a-db56447c87a4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:33.896726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:33.897689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:33.898569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 480/994] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:33.903851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fca7db-b2ea-49ef-b200-6fb4660f6942 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:33.980711 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1beb51-1726-44a9-999a-db56447c87a4 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:33.994880 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:34.009483 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fca7db-b2ea-49ef-b200-6fb4660f6942 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:34.044804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240698ee-23a5-4ea3-bf2c-7f80cf4cf75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:34.045504 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240698ee-23a5-4ea3-bf2c-7f80cf4cf75b None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:34.045775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240698ee-23a5-4ea3-bf2c-7f80cf4cf75b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:34.046020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240698ee-23a5-4ea3-bf2c-7f80cf4cf75b None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:34.046625 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240698ee-23a5-4ea3-bf2c-7f80cf4cf75b None admin] RBAC: Adding query filter params (name=compute) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:34.046930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240698ee-23a5-4ea3-bf2c-7f80cf4cf75b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:34.048406 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240698ee-23a5-4ea3-bf2c-7f80cf4cf75b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:34.055171 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 515/995] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:34.059620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240698ee-23a5-4ea3-bf2c-7f80cf4cf75b None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:34.096408 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:34.102919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1beb51-1726-44a9-999a-db56447c87a4 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:34.137088 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de62d5-486d-4799-b77c-3896725d3620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:34.138050 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de62d5-486d-4799-b77c-3896725d3620 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:34.138325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de62d5-486d-4799-b77c-3896725d3620 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:34.138592 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de62d5-486d-4799-b77c-3896725d3620 None admin] PATH_INFO: `/v3/services/image` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:34.139207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de62d5-486d-4799-b77c-3896725d3620 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:34.141266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de62d5-486d-4799-b77c-3896725d3620 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-73de62d5-486d-4799-b77c-3896725d3620 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:34.150603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:34.151644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:34.152532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 481/996] 10.222.0.16 () {56 vars in 1105 bytes} [Tue Aug 30 11:37:33 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:34.157746 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de62d5-486d-4799-b77c-3896725d3620 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:34.165634 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240698ee-23a5-4ea3-bf2c-7f80cf4cf75b None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:34.209239 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eec5e9-fff3-4647-b3b0-8d0746ae84c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:34.210046 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eec5e9-fff3-4647-b3b0-8d0746ae84c0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:34.210698 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eec5e9-fff3-4647-b3b0-8d0746ae84c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:34.210994 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eec5e9-fff3-4647-b3b0-8d0746ae84c0 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:34.211908 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eec5e9-fff3-4647-b3b0-8d0746ae84c0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:34.214444 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eec5e9-fff3-4647-b3b0-8d0746ae84c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:34.216152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eec5e9-fff3-4647-b3b0-8d0746ae84c0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:34.227080 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 516/997] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:34.228997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eec5e9-fff3-4647-b3b0-8d0746ae84c0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:34.245570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:34.273089 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de62d5-486d-4799-b77c-3896725d3620 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:34.308592 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daca35-24ba-455c-bb67-f2c250b0fbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:34.309265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daca35-24ba-455c-bb67-f2c250b0fbe7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:34.309539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daca35-24ba-455c-bb67-f2c250b0fbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:34.309819 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daca35-24ba-455c-bb67-f2c250b0fbe7 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:34.310433 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daca35-24ba-455c-bb67-f2c250b0fbe7 None admin] RBAC: Adding query filter params (name=image) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:34.310733 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daca35-24ba-455c-bb67-f2c250b0fbe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:34.312147 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daca35-24ba-455c-bb67-f2c250b0fbe7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:34.319170 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 482/998] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:34.322725 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daca35-24ba-455c-bb67-f2c250b0fbe7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:34.346085 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:34.350091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eec5e9-fff3-4647-b3b0-8d0746ae84c0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:34.385673 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fe306-cee0-4fe9-b117-a352c2ecf2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:34.386195 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fe306-cee0-4fe9-b117-a352c2ecf2e1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:34.386469 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fe306-cee0-4fe9-b117-a352c2ecf2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:34.386768 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fe306-cee0-4fe9-b117-a352c2ecf2e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:34.387506 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fe306-cee0-4fe9-b117-a352c2ecf2e1 None admin] RBAC: Adding query filter params (interface=public, service_id=3ebd00f2a0aa4adb92c6aa5fb8355b4f, region_id=RegionOne) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:34.387754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fe306-cee0-4fe9-b117-a352c2ecf2e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:34.389122 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fe306-cee0-4fe9-b117-a352c2ecf2e1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:34.398745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 517/999] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Aug 30 11:37:34 2022] GET /identity/v3/endpoints?service_id=3ebd00f2a0aa4adb92c6aa5fb8355b4f&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:34.399818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fe306-cee0-4fe9-b117-a352c2ecf2e1 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:34.440192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daca35-24ba-455c-bb67-f2c250b0fbe7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:34.484080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75458e38-deab-4639-98ef-424da6800fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:34.485059 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458e38-deab-4639-98ef-424da6800fea None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:34.485561 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458e38-deab-4639-98ef-424da6800fea None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:34.486020 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458e38-deab-4639-98ef-424da6800fea None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:34.486851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75458e38-deab-4639-98ef-424da6800fea None admin] RBAC: Adding query filter params (type=image) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:34.487343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75458e38-deab-4639-98ef-424da6800fea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:34.488907 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75458e38-deab-4639-98ef-424da6800fea None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:34.499484 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 483/1000] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:34.506741 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fe306-cee0-4fe9-b117-a352c2ecf2e1 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:34.546520 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b5401-903b-4c28-a801-d620032865dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:34.547448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b5401-903b-4c28-a801-d620032865dc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:34.547725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b5401-903b-4c28-a801-d620032865dc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:34.547993 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b5401-903b-4c28-a801-d620032865dc None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:34.548606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b5401-903b-4c28-a801-d620032865dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:34.550074 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b5401-903b-4c28-a801-d620032865dc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:34.557458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 518/1001] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Aug 30 11:37:34 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 11:37:34.767191 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef77b0-4dfb-4127-baf1-96bb6f35fc88 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:34.767523 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef77b0-4dfb-4127-baf1-96bb6f35fc88 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:34.767812 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef77b0-4dfb-4127-baf1-96bb6f35fc88 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:34.769243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 484/1002] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:34.774729 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e84d0f-e317-46e2-8b3b-9efc395f9277 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:34.774977 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e84d0f-e317-46e2-8b3b-9efc395f9277 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:34.775318 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e84d0f-e317-46e2-8b3b-9efc395f9277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:34.794297 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-27e84d0f-e317-46e2-8b3b-9efc395f9277 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:34.795525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-27e84d0f-e317-46e2-8b3b-9efc395f9277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:34.820909 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:34.870551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 519/1003] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:35.133059 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef77b0-4dfb-4127-baf1-96bb6f35fc88 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:35.136093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:35.240181 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef77b0-4dfb-4127-baf1-96bb6f35fc88 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:35.272861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11906502-07a3-434b-9c3b-0f4024c18c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:35.273487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11906502-07a3-434b-9c3b-0f4024c18c73 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:35.273763 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11906502-07a3-434b-9c3b-0f4024c18c73 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:35.274027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11906502-07a3-434b-9c3b-0f4024c18c73 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:35.274606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11906502-07a3-434b-9c3b-0f4024c18c73 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:35.275988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11906502-07a3-434b-9c3b-0f4024c18c73 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-11906502-07a3-434b-9c3b-0f4024c18c73 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:35.284729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:35.285701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:35.288243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 485/1004] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Aug 30 11:37:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:35.291881 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e84d0f-e317-46e2-8b3b-9efc395f9277 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:35.390612 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e84d0f-e317-46e2-8b3b-9efc395f9277 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:35.429482 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287f35e-d31b-463a-aabd-3d5f2294d346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:35.430096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287f35e-d31b-463a-aabd-3d5f2294d346 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:35.430367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287f35e-d31b-463a-aabd-3d5f2294d346 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:35.430627 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287f35e-d31b-463a-aabd-3d5f2294d346 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:35.431277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287f35e-d31b-463a-aabd-3d5f2294d346 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:35.431577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287f35e-d31b-463a-aabd-3d5f2294d346 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:35.432985 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287f35e-d31b-463a-aabd-3d5f2294d346 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:35.441954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 520/1005] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:35.447630 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11906502-07a3-434b-9c3b-0f4024c18c73 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:35.516418 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:35.551772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11906502-07a3-434b-9c3b-0f4024c18c73 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:35.587669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a37c14-a6f6-4166-8a0a-cf8bbb6fcdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:35.588301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a37c14-a6f6-4166-8a0a-cf8bbb6fcdb3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:35.588573 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a37c14-a6f6-4166-8a0a-cf8bbb6fcdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:35.588838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a37c14-a6f6-4166-8a0a-cf8bbb6fcdb3 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:35.589484 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a37c14-a6f6-4166-8a0a-cf8bbb6fcdb3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:35.589784 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a37c14-a6f6-4166-8a0a-cf8bbb6fcdb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:35.591219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a37c14-a6f6-4166-8a0a-cf8bbb6fcdb3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:35.597107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 486/1006] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Aug 30 11:37:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:35.602760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287f35e-d31b-463a-aabd-3d5f2294d346 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:35.630211 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:35.710002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287f35e-d31b-463a-aabd-3d5f2294d346 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:35.743013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f899281-fb27-465e-b3c9-fe6293c75e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:35.743820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f899281-fb27-465e-b3c9-fe6293c75e0f None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:35.743936 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f899281-fb27-465e-b3c9-fe6293c75e0f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:35.744169 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f899281-fb27-465e-b3c9-fe6293c75e0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:35.744833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f899281-fb27-465e-b3c9-fe6293c75e0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:35.746696 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f899281-fb27-465e-b3c9-fe6293c75e0f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:35.762766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 521/1007] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:37:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:37:36.397559 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c97a0-6c47-4f3f-9153-3c31d3a24111 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:36.397890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c97a0-6c47-4f3f-9153-3c31d3a24111 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:36.398175 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c97a0-6c47-4f3f-9153-3c31d3a24111 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:36.399690 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 487/1008] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:36.405004 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe622d1-2c3e-4bc4-a299-1ae8e669ae5d None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:36.405479 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe622d1-2c3e-4bc4-a299-1ae8e669ae5d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:36.405568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe622d1-2c3e-4bc4-a299-1ae8e669ae5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:36.427843 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-dfe622d1-2c3e-4bc4-a299-1ae8e669ae5d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:36.429035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-dfe622d1-2c3e-4bc4-a299-1ae8e669ae5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:36.442497 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd611a0-a278-42ab-a045-7c2313672f1b None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:36.442772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd611a0-a278-42ab-a045-7c2313672f1b None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:36.443070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd611a0-a278-42ab-a045-7c2313672f1b None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:36.444520 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 488/1009] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:36.449306 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b26cf0-6365-43d3-8e82-12ae34fcd20c None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:36.449823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b26cf0-6365-43d3-8e82-12ae34fcd20c None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:36.449823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b26cf0-6365-43d3-8e82-12ae34fcd20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:36.474250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-b2b26cf0-6365-43d3-8e82-12ae34fcd20c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:36.475881 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-b2b26cf0-6365-43d3-8e82-12ae34fcd20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:36.505498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 522/1010] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:36.549457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 489/1011] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:36.765475 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe622d1-2c3e-4bc4-a299-1ae8e669ae5d None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:36.776546 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:36.809902 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b26cf0-6365-43d3-8e82-12ae34fcd20c None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:36.869353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:36.875462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe622d1-2c3e-4bc4-a299-1ae8e669ae5d None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:36.908879 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b26cf0-6365-43d3-8e82-12ae34fcd20c None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:36.912008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f3652-b3cd-442f-99f3-0d213969333f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:36.912614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f3652-b3cd-442f-99f3-0d213969333f None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:36.912880 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f3652-b3cd-442f-99f3-0d213969333f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:36.913141 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f3652-b3cd-442f-99f3-0d213969333f None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:36.913788 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f3652-b3cd-442f-99f3-0d213969333f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:36.915238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f3652-b3cd-442f-99f3-0d213969333f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:36.932893 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 523/1012] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:37:36 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:36.950679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60467685-0b66-4264-bfc3-bc690783b242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:36.951353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60467685-0b66-4264-bfc3-bc690783b242 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:36.951629 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60467685-0b66-4264-bfc3-bc690783b242 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:36.951887 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60467685-0b66-4264-bfc3-bc690783b242 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:36.952479 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60467685-0b66-4264-bfc3-bc690783b242 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:36.954572 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60467685-0b66-4264-bfc3-bc690783b242 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-60467685-0b66-4264-bfc3-bc690783b242 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:36.965167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:36.966294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 11:37:36.967609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:36.971619 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 490/1013] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:36.975167 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f3652-b3cd-442f-99f3-0d213969333f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:37.051233 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:37.092743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f3652-b3cd-442f-99f3-0d213969333f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:37.127530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f05a8-dfb2-453a-844c-923a2b4f016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:37.128277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f05a8-dfb2-453a-844c-923a2b4f016d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:37.128560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f05a8-dfb2-453a-844c-923a2b4f016d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:37.128826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f05a8-dfb2-453a-844c-923a2b4f016d None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:37.129467 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f05a8-dfb2-453a-844c-923a2b4f016d None admin] RBAC: Adding query filter params (name=compute) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:37.129778 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f05a8-dfb2-453a-844c-923a2b4f016d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:37.131145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f05a8-dfb2-453a-844c-923a2b4f016d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:37.136554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 524/1014] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:37.142190 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60467685-0b66-4264-bfc3-bc690783b242 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:37.242633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60467685-0b66-4264-bfc3-bc690783b242 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:37.262682 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:37.277524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0f64d-a5a0-467c-8077-dbf57f0bf551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:37.278219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0f64d-a5a0-467c-8077-dbf57f0bf551 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:37.278561 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0f64d-a5a0-467c-8077-dbf57f0bf551 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:37.278893 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0f64d-a5a0-467c-8077-dbf57f0bf551 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:37.279623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0f64d-a5a0-467c-8077-dbf57f0bf551 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:37.280025 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0f64d-a5a0-467c-8077-dbf57f0bf551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:37.281494 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0f64d-a5a0-467c-8077-dbf57f0bf551 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:37.286842 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 491/1015] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Aug 30 11:37:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:37.292466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f05a8-dfb2-453a-844c-923a2b4f016d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:37.392648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f05a8-dfb2-453a-844c-923a2b4f016d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:37.427945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:37.434505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d8d10-2e97-4c3f-9b13-ca345953d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:37.438177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d8d10-2e97-4c3f-9b13-ca345953d4d3 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:37.438177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d8d10-2e97-4c3f-9b13-ca345953d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:37.438177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d8d10-2e97-4c3f-9b13-ca345953d4d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:37.438177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d8d10-2e97-4c3f-9b13-ca345953d4d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:37.438177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d8d10-2e97-4c3f-9b13-ca345953d4d3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:37.463379 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 525/1016] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:37:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:37.644806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b258701-603b-4342-94a0-8545e98cd021 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:37.645305 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b258701-603b-4342-94a0-8545e98cd021 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:37.645305 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b258701-603b-4342-94a0-8545e98cd021 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:37.646600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 492/1017] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:37.652826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e05c29-baf5-4e45-9c48-bebedc643297 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:37.653109 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e05c29-baf5-4e45-9c48-bebedc643297 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:37.653374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e05c29-baf5-4e45-9c48-bebedc643297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:37.672182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-86e05c29-baf5-4e45-9c48-bebedc643297 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:37.673216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-86e05c29-baf5-4e45-9c48-bebedc643297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:37.751744 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 526/1018] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:38.015629 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b258701-603b-4342-94a0-8545e98cd021 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:38.120343 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b258701-603b-4342-94a0-8545e98cd021 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:38.156457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9458fe5-5cf8-4248-a810-db014b39c83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:38.157092 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9458fe5-5cf8-4248-a810-db014b39c83a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:38.157364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9458fe5-5cf8-4248-a810-db014b39c83a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:38.157627 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9458fe5-5cf8-4248-a810-db014b39c83a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:38.158772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9458fe5-5cf8-4248-a810-db014b39c83a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:38.160302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9458fe5-5cf8-4248-a810-db014b39c83a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:38.162055 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 493/1019] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:38.167766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e05c29-baf5-4e45-9c48-bebedc643297 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:38.266799 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e05c29-baf5-4e45-9c48-bebedc643297 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:38.280797 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:38.301949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857af397-e2b3-4699-9d16-1b625ebe6f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:38.302561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857af397-e2b3-4699-9d16-1b625ebe6f57 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:38.302829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857af397-e2b3-4699-9d16-1b625ebe6f57 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:38.303091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857af397-e2b3-4699-9d16-1b625ebe6f57 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:38.303766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857af397-e2b3-4699-9d16-1b625ebe6f57 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:38.305190 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857af397-e2b3-4699-9d16-1b625ebe6f57 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:38.332852 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 527/1020] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:37:38 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:38.835576 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4368c-3b2d-4eff-80a8-8ba8fe219739 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:38.836127 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4368c-3b2d-4eff-80a8-8ba8fe219739 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:38.836127 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4368c-3b2d-4eff-80a8-8ba8fe219739 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:38.837834 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 494/1021] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:38.843864 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73a9e2-fc93-4dbc-b4d6-ac61cae5f3db None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:38.844346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73a9e2-fc93-4dbc-b4d6-ac61cae5f3db None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:38.844346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73a9e2-fc93-4dbc-b4d6-ac61cae5f3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:38.862553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-2b73a9e2-fc93-4dbc-b4d6-ac61cae5f3db None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:38.863750 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-2b73a9e2-fc93-4dbc-b4d6-ac61cae5f3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:38.933367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 528/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:39.206495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4368c-3b2d-4eff-80a8-8ba8fe219739 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:39.261979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:39.311986 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4368c-3b2d-4eff-80a8-8ba8fe219739 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:39.345944 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c12acb-6550-4d76-b8bd-7bc723c73907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:39.346581 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c12acb-6550-4d76-b8bd-7bc723c73907 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:39.346909 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c12acb-6550-4d76-b8bd-7bc723c73907 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:39.347207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c12acb-6550-4d76-b8bd-7bc723c73907 None admin] PATH_INFO: `/v3/services/image` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:39.347794 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c12acb-6550-4d76-b8bd-7bc723c73907 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:39.349166 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c12acb-6550-4d76-b8bd-7bc723c73907 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-f2c12acb-6550-4d76-b8bd-7bc723c73907 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:39.358353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:39.363406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:39.364297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 495/1023] 10.222.0.16 () {56 vars in 1105 bytes} [Tue Aug 30 11:37:39 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:39.365530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73a9e2-fc93-4dbc-b4d6-ac61cae5f3db None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:39.442552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59500f35-204b-4983-bd5a-67fbbe34c357 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:39.442838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59500f35-204b-4983-bd5a-67fbbe34c357 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:39.443128 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59500f35-204b-4983-bd5a-67fbbe34c357 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:39.444916 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 496/1024] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:39.449644 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c9b99-10ba-4c65-bc36-933a75b9ea7d None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:39.449772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c9b99-10ba-4c65-bc36-933a75b9ea7d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:39.449964 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c9b99-10ba-4c65-bc36-933a75b9ea7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:39.468152 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-216c9b99-10ba-4c65-bc36-933a75b9ea7d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:39.469253 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-216c9b99-10ba-4c65-bc36-933a75b9ea7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:39.471311 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:39.478741 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73a9e2-fc93-4dbc-b4d6-ac61cae5f3db None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:39.514597 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e10b97-bf44-4563-84d2-61ae6fde07be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:39.515253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e10b97-bf44-4563-84d2-61ae6fde07be None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:39.515527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e10b97-bf44-4563-84d2-61ae6fde07be None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:39.515793 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e10b97-bf44-4563-84d2-61ae6fde07be None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:39.516420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e10b97-bf44-4563-84d2-61ae6fde07be None admin] RBAC: Adding query filter params (name=image) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:39.516718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e10b97-bf44-4563-84d2-61ae6fde07be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:39.518107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e10b97-bf44-4563-84d2-61ae6fde07be None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:39.528092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 529/1025] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:39.529446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e10b97-bf44-4563-84d2-61ae6fde07be None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:39.539153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 497/1026] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:39.633102 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e10b97-bf44-4563-84d2-61ae6fde07be None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:39.667833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5a340-a403-4a59-821f-85aadaa28cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:39.668443 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5a340-a403-4a59-821f-85aadaa28cfa None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:39.668758 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5a340-a403-4a59-821f-85aadaa28cfa None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:39.669030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5a340-a403-4a59-821f-85aadaa28cfa None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:39.669652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5a340-a403-4a59-821f-85aadaa28cfa None admin] RBAC: Adding query filter params (type=image) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:39.669950 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5a340-a403-4a59-821f-85aadaa28cfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:39.671379 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5a340-a403-4a59-821f-85aadaa28cfa None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:39.681008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 530/1027] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:39.682273 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c9b99-10ba-4c65-bc36-933a75b9ea7d None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:39.728045 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:39.782940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c9b99-10ba-4c65-bc36-933a75b9ea7d None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:39.802689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5a340-a403-4a59-821f-85aadaa28cfa None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:39.820338 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea03138-b31a-44ac-a1fa-aaa2e3bd3a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:39.820952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea03138-b31a-44ac-a1fa-aaa2e3bd3a66 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:39.821224 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea03138-b31a-44ac-a1fa-aaa2e3bd3a66 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:39.821468 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea03138-b31a-44ac-a1fa-aaa2e3bd3a66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:39.822183 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea03138-b31a-44ac-a1fa-aaa2e3bd3a66 None admin] RBAC: Adding query filter params (interface=public, service_id=20d9a3334c1d471483828f1ebab4b3c0, region_id=RegionOne) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:39.822488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea03138-b31a-44ac-a1fa-aaa2e3bd3a66 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:39.823974 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea03138-b31a-44ac-a1fa-aaa2e3bd3a66 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:39.828823 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:39.834260 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 498/1028] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Aug 30 11:37:39 2022] GET /identity/v3/endpoints?service_id=20d9a3334c1d471483828f1ebab4b3c0&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:39.835256 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea03138-b31a-44ac-a1fa-aaa2e3bd3a66 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:39.907828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5a340-a403-4a59-821f-85aadaa28cfa None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:39.937787 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea03138-b31a-44ac-a1fa-aaa2e3bd3a66 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:39.945071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d7004-d8ab-4d95-b6c8-4bfc8af814ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:39.946143 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d7004-d8ab-4d95-b6c8-4bfc8af814ec None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:39.946422 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d7004-d8ab-4d95-b6c8-4bfc8af814ec None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:39.946690 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d7004-d8ab-4d95-b6c8-4bfc8af814ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:39.948778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:39.960461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d7004-d8ab-4d95-b6c8-4bfc8af814ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-f26d7004-d8ab-4d95-b6c8-4bfc8af814ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:39.965591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:39.966549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:39.967468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 531/1029] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Aug 30 11:37:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:39.973333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d7004-d8ab-4d95-b6c8-4bfc8af814ec None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:39.976720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbbe85-0ee7-429b-b016-c60d6af6aef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:39.977330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbbe85-0ee7-429b-b016-c60d6af6aef7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:39.977604 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbbe85-0ee7-429b-b016-c60d6af6aef7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:39.977874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbbe85-0ee7-429b-b016-c60d6af6aef7 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:39.978533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbbe85-0ee7-429b-b016-c60d6af6aef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:39.979873 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbbe85-0ee7-429b-b016-c60d6af6aef7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:39.986676 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 499/1030] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Aug 30 11:37:39 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 11:37:40.067272 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:40.084825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d7004-d8ab-4d95-b6c8-4bfc8af814ec None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:40.121644 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341af79-8d76-4693-93c5-040ca086d77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:40.122254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341af79-8d76-4693-93c5-040ca086d77d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.122525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341af79-8d76-4693-93c5-040ca086d77d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.122788 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341af79-8d76-4693-93c5-040ca086d77d None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:40.123596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341af79-8d76-4693-93c5-040ca086d77d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:40.123894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341af79-8d76-4693-93c5-040ca086d77d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:40.125281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341af79-8d76-4693-93c5-040ca086d77d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:40.134129 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 532/1031] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Aug 30 11:37:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:40.140828 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbbe85-0ee7-429b-b016-c60d6af6aef7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:40.239368 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916b626-8b35-47c7-8f71-cf2d2132fe59 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.239574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916b626-8b35-47c7-8f71-cf2d2132fe59 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.239807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916b626-8b35-47c7-8f71-cf2d2132fe59 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:40.241372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 533/1032] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:40.246031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48955b83-3f53-48ed-8f26-37c411dac241 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.246306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48955b83-3f53-48ed-8f26-37c411dac241 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.246572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48955b83-3f53-48ed-8f26-37c411dac241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:40.246867 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbbe85-0ee7-429b-b016-c60d6af6aef7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:40.264305 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-48955b83-3f53-48ed-8f26-37c411dac241 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:40.265476 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-48955b83-3f53-48ed-8f26-37c411dac241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:40.287260 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c30ab-3ba8-4b56-9042-768f2b2b59fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:40.288175 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c30ab-3ba8-4b56-9042-768f2b2b59fd None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.288453 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c30ab-3ba8-4b56-9042-768f2b2b59fd None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.288719 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c30ab-3ba8-4b56-9042-768f2b2b59fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:40.299659 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c30ab-3ba8-4b56-9042-768f2b2b59fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:40.301148 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c30ab-3ba8-4b56-9042-768f2b2b59fd None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-eb5c30ab-3ba8-4b56-9042-768f2b2b59fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:40.316078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:40.317077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:40.318005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:40.318005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:40.318005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:40.318005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 500/1033] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:40.322880 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c30ab-3ba8-4b56-9042-768f2b2b59fd None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:40.336677 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 534/1034] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:40.352026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:40.429824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c30ab-3ba8-4b56-9042-768f2b2b59fd None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:40.465558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff930eb-15fe-4d88-b0e8-802542a0a4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:40.466169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff930eb-15fe-4d88-b0e8-802542a0a4b1 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.466444 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff930eb-15fe-4d88-b0e8-802542a0a4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.466714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff930eb-15fe-4d88-b0e8-802542a0a4b1 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:40.467502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff930eb-15fe-4d88-b0e8-802542a0a4b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:40.467812 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff930eb-15fe-4d88-b0e8-802542a0a4b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:40.469181 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff930eb-15fe-4d88-b0e8-802542a0a4b1 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:40.485823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 501/1035] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:40.487461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48955b83-3f53-48ed-8f26-37c411dac241 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:40.564619 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:40.589848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48955b83-3f53-48ed-8f26-37c411dac241 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:40.597723 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff930eb-15fe-4d88-b0e8-802542a0a4b1 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:40.626341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef1a2e-fdd8-4a97-b545-6a84f68c510e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:40.626953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef1a2e-fdd8-4a97-b545-6a84f68c510e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.627254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef1a2e-fdd8-4a97-b545-6a84f68c510e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.627518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef1a2e-fdd8-4a97-b545-6a84f68c510e None admin] PATH_INFO: `/v3/users/nova` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:40.634869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef1a2e-fdd8-4a97-b545-6a84f68c510e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:40.636343 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef1a2e-fdd8-4a97-b545-6a84f68c510e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-89ef1a2e-fdd8-4a97-b545-6a84f68c510e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:40.647820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:40.648857 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:40.649830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 535/1036] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Aug 30 11:37:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:40.655395 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef1a2e-fdd8-4a97-b545-6a84f68c510e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:40.701991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:40.715824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff930eb-15fe-4d88-b0e8-802542a0a4b1 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:40.751404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c5386-300f-42f1-8b42-8bfa81baefce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:40.752026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c5386-300f-42f1-8b42-8bfa81baefce None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.752301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c5386-300f-42f1-8b42-8bfa81baefce None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.752556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c5386-300f-42f1-8b42-8bfa81baefce None admin] PATH_INFO: `/v3/domains/default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:40.753664 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c5386-300f-42f1-8b42-8bfa81baefce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:40.755086 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c5386-300f-42f1-8b42-8bfa81baefce None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:40.756835 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 502/1037] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:40.762166 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c5386-300f-42f1-8b42-8bfa81baefce None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:40.769036 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef1a2e-fdd8-4a97-b545-6a84f68c510e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:40.803365 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf581df-3876-40f9-a740-800541d4913e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:40.803992 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf581df-3876-40f9-a740-800541d4913e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.804263 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf581df-3876-40f9-a740-800541d4913e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.804526 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf581df-3876-40f9-a740-800541d4913e None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:40.805235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf581df-3876-40f9-a740-800541d4913e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:40.805530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf581df-3876-40f9-a740-800541d4913e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:40.806985 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf581df-3876-40f9-a740-800541d4913e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:40.810028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:40.829463 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 536/1038] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Aug 30 11:37:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:40.835035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf581df-3876-40f9-a740-800541d4913e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:40.870972 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c5386-300f-42f1-8b42-8bfa81baefce None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:40.907832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67f80d-c32c-4eea-a00c-29ac40118aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:40.908465 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67f80d-c32c-4eea-a00c-29ac40118aac None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.908776 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67f80d-c32c-4eea-a00c-29ac40118aac None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.909044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67f80d-c32c-4eea-a00c-29ac40118aac None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:40.909730 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67f80d-c32c-4eea-a00c-29ac40118aac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:40.911274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67f80d-c32c-4eea-a00c-29ac40118aac None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:40.943220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 503/1039] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:37:40 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:37:40.943884 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf581df-3876-40f9-a740-800541d4913e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:40.983088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2ae10-4dfc-4d0e-b0ab-10257575ecd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:40.983832 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2ae10-4dfc-4d0e-b0ab-10257575ecd2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:40.984108 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2ae10-4dfc-4d0e-b0ab-10257575ecd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:40.984376 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2ae10-4dfc-4d0e-b0ab-10257575ecd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:41.051446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:41.062075 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2ae10-4dfc-4d0e-b0ab-10257575ecd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:41.063634 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2ae10-4dfc-4d0e-b0ab-10257575ecd2 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:41.171928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 537/1040] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:37:41.177287 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67f80d-c32c-4eea-a00c-29ac40118aac None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:41.190582 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:41.280189 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67f80d-c32c-4eea-a00c-29ac40118aac None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:41.313164 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2a148-169a-4036-80b0-b8b22059644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:41.313791 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2a148-169a-4036-80b0-b8b22059644d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:41.314058 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2a148-169a-4036-80b0-b8b22059644d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:41.314322 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2a148-169a-4036-80b0-b8b22059644d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:41.325102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2a148-169a-4036-80b0-b8b22059644d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:41.326545 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2a148-169a-4036-80b0-b8b22059644d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-95e2a148-169a-4036-80b0-b8b22059644d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:41.340859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:41.341824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:41.343138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:41.343138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:41.343138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:41.343138 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 504/1041] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:41.347942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2ae10-4dfc-4d0e-b0ab-10257575ecd2 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:41.456582 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:41.465642 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2ae10-4dfc-4d0e-b0ab-10257575ecd2 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:41.517845 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c948a-771e-4ff4-8806-aade7518aa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:41.518886 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c948a-771e-4ff4-8806-aade7518aa5c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:41.519394 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c948a-771e-4ff4-8806-aade7518aa5c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:41.519860 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c948a-771e-4ff4-8806-aade7518aa5c None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:41.520985 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c948a-771e-4ff4-8806-aade7518aa5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:41.521506 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c948a-771e-4ff4-8806-aade7518aa5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:41.524061 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c948a-771e-4ff4-8806-aade7518aa5c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:41.543799 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 538/1042] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:41.549984 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2a148-169a-4036-80b0-b8b22059644d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:41.613262 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:41.656292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2a148-169a-4036-80b0-b8b22059644d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:41.691127 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8206f-b614-40a2-92c2-a59bd33e3bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:41.691683 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8206f-b614-40a2-92c2-a59bd33e3bcc None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:41.691926 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8206f-b614-40a2-92c2-a59bd33e3bcc None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:41.692222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8206f-b614-40a2-92c2-a59bd33e3bcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:41.698930 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8206f-b614-40a2-92c2-a59bd33e3bcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:41.700416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8206f-b614-40a2-92c2-a59bd33e3bcc None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-31a8206f-b614-40a2-92c2-a59bd33e3bcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:41.711919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:41.712807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:41.713695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:41.713695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:41.713695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:41.713695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:41.713695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:41.713695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:41.713695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:41.713695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:41.714919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 505/1043] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:41.718883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c948a-771e-4ff4-8806-aade7518aa5c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:41.821723 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c948a-771e-4ff4-8806-aade7518aa5c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:41.855388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e929be7-3b74-4042-9e5a-3fd4eb36ff43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:41.855996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e929be7-3b74-4042-9e5a-3fd4eb36ff43 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:41.856268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e929be7-3b74-4042-9e5a-3fd4eb36ff43 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:41.856529 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e929be7-3b74-4042-9e5a-3fd4eb36ff43 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:41.857258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e929be7-3b74-4042-9e5a-3fd4eb36ff43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:41.857591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e929be7-3b74-4042-9e5a-3fd4eb36ff43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:41.859098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e929be7-3b74-4042-9e5a-3fd4eb36ff43 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:41.871230 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 539/1044] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:41.877375 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8206f-b614-40a2-92c2-a59bd33e3bcc None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:41.891460 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f2ad6-fdec-40e0-b02e-67deb094f37b None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:41.891756 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f2ad6-fdec-40e0-b02e-67deb094f37b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:41.892014 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f2ad6-fdec-40e0-b02e-67deb094f37b None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:41.893492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 540/1045] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:41.898147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d88a22-3c1c-4174-82a3-363a5c3aab3c None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:41.898390 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d88a22-3c1c-4174-82a3-363a5c3aab3c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:41.898681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d88a22-3c1c-4174-82a3-363a5c3aab3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:41.916803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-69d88a22-3c1c-4174-82a3-363a5c3aab3c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:41.917969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-69d88a22-3c1c-4174-82a3-363a5c3aab3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:41.928191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:41.984590 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8206f-b614-40a2-92c2-a59bd33e3bcc None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:41.987525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 541/1046] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:42.008496 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:42.022031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f4477-446c-4b2e-a61c-8f0a47a17397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:42.022631 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f4477-446c-4b2e-a61c-8f0a47a17397 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:42.022891 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f4477-446c-4b2e-a61c-8f0a47a17397 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:42.023174 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f4477-446c-4b2e-a61c-8f0a47a17397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:42.023910 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f4477-446c-4b2e-a61c-8f0a47a17397 None admin] RBAC: Adding query filter params (role.id=64b48b964be3471a89c5b9cf4fc9b990, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=0bb80229c3234d5f9599b16870c69934) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:42.024211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f4477-446c-4b2e-a61c-8f0a47a17397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:42.025634 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f4477-446c-4b2e-a61c-8f0a47a17397 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:42.031415 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 506/1047] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:41 2022] GET /identity/v3/role_assignments?user.id=0bb80229c3234d5f9599b16870c69934&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=64b48b964be3471a89c5b9cf4fc9b990 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:42.251816 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d88a22-3c1c-4174-82a3-363a5c3aab3c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:42.353373 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d88a22-3c1c-4174-82a3-363a5c3aab3c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:42.388543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b1ae-43fc-47a1-b8f7-348e37071ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:42.389185 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b1ae-43fc-47a1-b8f7-348e37071ad0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:42.389461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b1ae-43fc-47a1-b8f7-348e37071ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:42.389733 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b1ae-43fc-47a1-b8f7-348e37071ad0 None admin] PATH_INFO: `/v3/services/image` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:42.390292 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8b1ae-43fc-47a1-b8f7-348e37071ad0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:42.391700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8b1ae-43fc-47a1-b8f7-348e37071ad0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-bed8b1ae-43fc-47a1-b8f7-348e37071ad0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:42.400660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:42.401648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 11:37:42.402547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:42.406788 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 542/1048] 10.222.0.16 () {56 vars in 1105 bytes} [Tue Aug 30 11:37:42 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:42.410703 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f4477-446c-4b2e-a61c-8f0a47a17397 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:42.519360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f4477-446c-4b2e-a61c-8f0a47a17397 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:42.552712 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f799074-119b-4ab4-89de-2633b7f110b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:42.553340 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f799074-119b-4ab4-89de-2633b7f110b3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:42.553618 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f799074-119b-4ab4-89de-2633b7f110b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:42.553882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f799074-119b-4ab4-89de-2633b7f110b3 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:42.554507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f799074-119b-4ab4-89de-2633b7f110b3 None admin] RBAC: Adding query filter params (name=image) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:42.554808 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f799074-119b-4ab4-89de-2633b7f110b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:42.556231 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f799074-119b-4ab4-89de-2633b7f110b3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:42.561242 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 507/1049] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:42.566896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b1ae-43fc-47a1-b8f7-348e37071ad0 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:42.578357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:42.667995 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b1ae-43fc-47a1-b8f7-348e37071ad0 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:42.701082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12326d9-f130-4f1f-bb20-b7901c8b8d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:42.701689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12326d9-f130-4f1f-bb20-b7901c8b8d3b None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:42.701958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12326d9-f130-4f1f-bb20-b7901c8b8d3b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:42.702224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12326d9-f130-4f1f-bb20-b7901c8b8d3b None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:42.702848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12326d9-f130-4f1f-bb20-b7901c8b8d3b None admin] RBAC: Adding query filter params (type=image) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:42.703157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12326d9-f130-4f1f-bb20-b7901c8b8d3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:42.704869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12326d9-f130-4f1f-bb20-b7901c8b8d3b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:42.709869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 543/1050] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:42.715495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f799074-119b-4ab4-89de-2633b7f110b3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:42.764793 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:42.815995 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f799074-119b-4ab4-89de-2633b7f110b3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:42.854514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c9d02-db9b-4432-ac57-740f81a0b463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:42.855257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c9d02-db9b-4432-ac57-740f81a0b463 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:42.855533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c9d02-db9b-4432-ac57-740f81a0b463 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:42.855809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c9d02-db9b-4432-ac57-740f81a0b463 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:42.856498 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c9d02-db9b-4432-ac57-740f81a0b463 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:42.857871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c9d02-db9b-4432-ac57-740f81a0b463 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:42.880346 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 508/1051] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:37:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:42.937059 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40637a4b-36c0-4b82-90e6-005103372a5d None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:42.937310 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40637a4b-36c0-4b82-90e6-005103372a5d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:42.937594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40637a4b-36c0-4b82-90e6-005103372a5d None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:42.939182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 544/1052] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:42.944990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43de1c-b3d2-4eaf-abec-1b3dc155a2a6 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:42.945333 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43de1c-b3d2-4eaf-abec-1b3dc155a2a6 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:42.945519 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43de1c-b3d2-4eaf-abec-1b3dc155a2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:42.966550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-3a43de1c-b3d2-4eaf-abec-1b3dc155a2a6 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:42.967656 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-3a43de1c-b3d2-4eaf-abec-1b3dc155a2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:43.051661 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 509/1053] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:43.315826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40637a4b-36c0-4b82-90e6-005103372a5d None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:43.395466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:43.432052 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40637a4b-36c0-4b82-90e6-005103372a5d None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:43.467704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cdd7f-07d0-4013-b1a6-0567e60f327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:43.468351 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cdd7f-07d0-4013-b1a6-0567e60f327e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:43.468627 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cdd7f-07d0-4013-b1a6-0567e60f327e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:43.468896 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cdd7f-07d0-4013-b1a6-0567e60f327e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:43.479883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cdd7f-07d0-4013-b1a6-0567e60f327e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-aa4cdd7f-07d0-4013-b1a6-0567e60f327e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:43.484674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:43.485592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:43.486445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:43.488097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 545/1054] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:43.492024 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43de1c-b3d2-4eaf-abec-1b3dc155a2a6 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:43.524218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:43.596187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43de1c-b3d2-4eaf-abec-1b3dc155a2a6 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:43.644803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222febed-78f2-4153-8ce3-000ff61d5dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:43.644803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222febed-78f2-4153-8ce3-000ff61d5dd7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:43.644803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222febed-78f2-4153-8ce3-000ff61d5dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:43.644803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222febed-78f2-4153-8ce3-000ff61d5dd7 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:43.645252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222febed-78f2-4153-8ce3-000ff61d5dd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:43.645252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222febed-78f2-4153-8ce3-000ff61d5dd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:43.645252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222febed-78f2-4153-8ce3-000ff61d5dd7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:43.664059 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cdd7f-07d0-4013-b1a6-0567e60f327e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:43.664824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 510/1055] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:37:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:43.778718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cdd7f-07d0-4013-b1a6-0567e60f327e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:43.812349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:43.818784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2593ed-ca9a-4124-b38b-e1e0acfb9916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:43.819456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2593ed-ca9a-4124-b38b-e1e0acfb9916 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:43.819733 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2593ed-ca9a-4124-b38b-e1e0acfb9916 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:43.820002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2593ed-ca9a-4124-b38b-e1e0acfb9916 None admin] PATH_INFO: `/v3/users/bb90dd7aa98642bdae5c2daee7855503` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:43.835067 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2593ed-ca9a-4124-b38b-e1e0acfb9916 None admin] RBAC: Authorizing `identity:get_user(user_id=bb90dd7aa98642bdae5c2daee7855503)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:43.836588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2593ed-ca9a-4124-b38b-e1e0acfb9916 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:43.841837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 546/1056] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:37:43 2022] GET /identity/v3/users/bb90dd7aa98642bdae5c2daee7855503 => generated 285 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:43.844384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222febed-78f2-4153-8ce3-000ff61d5dd7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:43.884519 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047381e1-0b0a-49c1-8d2b-788b2150644e None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:43.884702 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047381e1-0b0a-49c1-8d2b-788b2150644e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:43.884960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047381e1-0b0a-49c1-8d2b-788b2150644e None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:43.886319 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 547/1057] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:37:43.891405 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad459673-d458-41e8-ba70-26ba4f5de43c None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:43.891681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad459673-d458-41e8-ba70-26ba4f5de43c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:43.892243 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad459673-d458-41e8-ba70-26ba4f5de43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:43.910885 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-ad459673-d458-41e8-ba70-26ba4f5de43c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:43.912295 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ad459673-d458-41e8-ba70-26ba4f5de43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:43.943674 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:43.951402 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222febed-78f2-4153-8ce3-000ff61d5dd7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:43.990230 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff6558-2a12-4950-97e8-f6eeca3df430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:43.991223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff6558-2a12-4950-97e8-f6eeca3df430 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:43.991508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 548/1058] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:37:43.991934 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff6558-2a12-4950-97e8-f6eeca3df430 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:43.992278 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff6558-2a12-4950-97e8-f6eeca3df430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:44.059753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff6558-2a12-4950-97e8-f6eeca3df430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:44.061201 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff6558-2a12-4950-97e8-f6eeca3df430 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:44.151197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:44.160369 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 511/1059] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:37:44.166365 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad459673-d458-41e8-ba70-26ba4f5de43c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:44.251822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff6558-2a12-4950-97e8-f6eeca3df430 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:44.278947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad459673-d458-41e8-ba70-26ba4f5de43c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:44.299775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:44.315658 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04588d8b-1733-4100-a96b-11a664c51506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:44.316297 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04588d8b-1733-4100-a96b-11a664c51506 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:44.316837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04588d8b-1733-4100-a96b-11a664c51506 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:44.316837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04588d8b-1733-4100-a96b-11a664c51506 None admin] PATH_INFO: `/v3/projects/889330087e064f19b3edbe84bf84e3ac` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:44.317941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04588d8b-1733-4100-a96b-11a664c51506 None admin] RBAC: Authorizing `identity:get_project(project_id=889330087e064f19b3edbe84bf84e3ac)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:44.319415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04588d8b-1733-4100-a96b-11a664c51506 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:44.321229 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 549/1060] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:37:44 2022] GET /identity/v3/projects/889330087e064f19b3edbe84bf84e3ac => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:44.327213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04588d8b-1733-4100-a96b-11a664c51506 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:44.358651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff6558-2a12-4950-97e8-f6eeca3df430 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:44.393221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92633537-5bde-40bd-929a-c36a8cb59ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:44.393902 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92633537-5bde-40bd-929a-c36a8cb59ec8 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:44.394173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92633537-5bde-40bd-929a-c36a8cb59ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:44.394441 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92633537-5bde-40bd-929a-c36a8cb59ec8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:44.404852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92633537-5bde-40bd-929a-c36a8cb59ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-92633537-5bde-40bd-929a-c36a8cb59ec8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:44.409624 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:44.410670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:44.411538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:44.415645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 512/1061] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Aug 30 11:37:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:44.416478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92633537-5bde-40bd-929a-c36a8cb59ec8 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:44.441663 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04588d8b-1733-4100-a96b-11a664c51506 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:44.476879 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:44.477992 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca9760-538f-4b84-b0bf-6efade90110f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:44.478996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9760-538f-4b84-b0bf-6efade90110f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:44.479490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9760-538f-4b84-b0bf-6efade90110f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:44.479877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9760-538f-4b84-b0bf-6efade90110f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:44.480490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca9760-538f-4b84-b0bf-6efade90110f None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=889330087e064f19b3edbe84bf84e3ac, user.id=bb90dd7aa98642bdae5c2daee7855503) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:44.480910 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca9760-538f-4b84-b0bf-6efade90110f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:44.482218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca9760-538f-4b84-b0bf-6efade90110f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:44.499153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 550/1062] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:44 2022] GET /identity/v3/role_assignments?user.id=bb90dd7aa98642bdae5c2daee7855503&scope.project.id=889330087e064f19b3edbe84bf84e3ac&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:44.538034 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92633537-5bde-40bd-929a-c36a8cb59ec8 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:44.576266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1927a-4475-4cc6-b769-76950abcccbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:44.576877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1927a-4475-4cc6-b769-76950abcccbe None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:44.577149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1927a-4475-4cc6-b769-76950abcccbe None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:44.577412 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1927a-4475-4cc6-b769-76950abcccbe None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:44.578076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1927a-4475-4cc6-b769-76950abcccbe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:44.578666 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1927a-4475-4cc6-b769-76950abcccbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:44.580093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1927a-4475-4cc6-b769-76950abcccbe None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:44.589342 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 513/1063] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Aug 30 11:37:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:44.595024 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca9760-538f-4b84-b0bf-6efade90110f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:44.652086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:44.703775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca9760-538f-4b84-b0bf-6efade90110f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:44.738351 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c951beb-ff3c-4c75-a7de-fd38f35fcb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:44.738980 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c951beb-ff3c-4c75-a7de-fd38f35fcb88 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:44.739278 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c951beb-ff3c-4c75-a7de-fd38f35fcb88 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:44.739540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c951beb-ff3c-4c75-a7de-fd38f35fcb88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:44.750337 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c951beb-ff3c-4c75-a7de-fd38f35fcb88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:44.752170 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c951beb-ff3c-4c75-a7de-fd38f35fcb88 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-8c951beb-ff3c-4c75-a7de-fd38f35fcb88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:44.766357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:44.767359 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:44.768290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:44.768290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:44.768290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:44.771207 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 551/1064] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:44.773465 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1927a-4475-4cc6-b769-76950abcccbe None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:44.781268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8197a1-9929-41f9-acc1-a1d9dfca9579 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:44.781542 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8197a1-9929-41f9-acc1-a1d9dfca9579 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:44.781814 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8197a1-9929-41f9-acc1-a1d9dfca9579 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:44.783323 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 552/1065] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:44.788448 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ce24c-13ea-44c1-aace-8826ca4bcd53 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:44.788727 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ce24c-13ea-44c1-aace-8826ca4bcd53 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:44.788994 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ce24c-13ea-44c1-aace-8826ca4bcd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:44.808604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-7e8ce24c-13ea-44c1-aace-8826ca4bcd53 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:44.809802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-7e8ce24c-13ea-44c1-aace-8826ca4bcd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:44.875670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1927a-4475-4cc6-b769-76950abcccbe None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:44.882394 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 553/1066] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:44.892235 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:44.910990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac52c77-c67f-4ff6-ba9a-e1fcbae9f1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:44.911699 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac52c77-c67f-4ff6-ba9a-e1fcbae9f1ae None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:44.911973 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac52c77-c67f-4ff6-ba9a-e1fcbae9f1ae None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:44.912240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac52c77-c67f-4ff6-ba9a-e1fcbae9f1ae None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:44.912887 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac52c77-c67f-4ff6-ba9a-e1fcbae9f1ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:44.913186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac52c77-c67f-4ff6-ba9a-e1fcbae9f1ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:44.914544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac52c77-c67f-4ff6-ba9a-e1fcbae9f1ae None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:44.927107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 514/1067] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:44.932833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ce24c-13ea-44c1-aace-8826ca4bcd53 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:44.969697 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:45.039626 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ce24c-13ea-44c1-aace-8826ca4bcd53 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:45.073568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd2e5-b159-4469-a0cb-057015e7d231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:45.074206 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd2e5-b159-4469-a0cb-057015e7d231 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:45.074491 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd2e5-b159-4469-a0cb-057015e7d231 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:45.074744 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd2e5-b159-4469-a0cb-057015e7d231 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:45.082399 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fd2e5-b159-4469-a0cb-057015e7d231 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:45.083873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fd2e5-b159-4469-a0cb-057015e7d231 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-363fd2e5-b159-4469-a0cb-057015e7d231 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:45.095468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:45.096396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:45.097374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:45.099470 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 554/1068] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Aug 30 11:37:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:45.103893 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac52c77-c67f-4ff6-ba9a-e1fcbae9f1ae None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:45.143550 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd2e5-b159-4469-a0cb-057015e7d231 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:45.211323 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac52c77-c67f-4ff6-ba9a-e1fcbae9f1ae None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:45.243082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:45.248568 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d8313-f86a-4b85-9e43-f7e65ab7ef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:45.249404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d8313-f86a-4b85-9e43-f7e65ab7ef63 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:45.249605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd2e5-b159-4469-a0cb-057015e7d231 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:45.250029 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d8313-f86a-4b85-9e43-f7e65ab7ef63 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:45.250463 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d8313-f86a-4b85-9e43-f7e65ab7ef63 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:45.251504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d8313-f86a-4b85-9e43-f7e65ab7ef63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:45.252043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d8313-f86a-4b85-9e43-f7e65ab7ef63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:45.253497 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d8313-f86a-4b85-9e43-f7e65ab7ef63 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:45.276362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 515/1069] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Aug 30 11:37:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:45.277317 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d8313-f86a-4b85-9e43-f7e65ab7ef63 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:45.287546 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5313a5-987f-450c-9cc9-88292afe4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:45.288160 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5313a5-987f-450c-9cc9-88292afe4196 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:45.288436 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5313a5-987f-450c-9cc9-88292afe4196 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:45.288670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5313a5-987f-450c-9cc9-88292afe4196 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:45.313050 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:45.356985 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5313a5-987f-450c-9cc9-88292afe4196 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:45.358732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5313a5-987f-450c-9cc9-88292afe4196 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:45.384469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d8313-f86a-4b85-9e43-f7e65ab7ef63 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:45.423235 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c96a79-8204-4413-8baa-3bf9952aa040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:45.423966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c96a79-8204-4413-8baa-3bf9952aa040 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:45.424243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c96a79-8204-4413-8baa-3bf9952aa040 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:45.424511 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c96a79-8204-4413-8baa-3bf9952aa040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:45.434649 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c96a79-8204-4413-8baa-3bf9952aa040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:45.436156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c96a79-8204-4413-8baa-3bf9952aa040 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-e9c96a79-8204-4413-8baa-3bf9952aa040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:45.454389 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:45.455423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:45.456472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:45.456472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:45.456472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:45.456472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 516/1070] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:45.458162 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:45.461565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c96a79-8204-4413-8baa-3bf9952aa040 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:45.471180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 555/1071] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 11:37:45.473010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5313a5-987f-450c-9cc9-88292afe4196 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:45.553707 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:45.570019 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c96a79-8204-4413-8baa-3bf9952aa040 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:45.582318 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5313a5-987f-450c-9cc9-88292afe4196 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:45.604333 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8f421-22d8-4149-9c23-47433a3d6308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:45.604938 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8f421-22d8-4149-9c23-47433a3d6308 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:45.605213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8f421-22d8-4149-9c23-47433a3d6308 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:45.605478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8f421-22d8-4149-9c23-47433a3d6308 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:45.606117 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8f421-22d8-4149-9c23-47433a3d6308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:45.606420 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8f421-22d8-4149-9c23-47433a3d6308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:45.607825 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8f421-22d8-4149-9c23-47433a3d6308 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:45.619748 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 517/1072] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:45.622535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7828-224b-4030-a02f-13b76e01d234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:45.623205 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7828-224b-4030-a02f-13b76e01d234 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:45.623484 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7828-224b-4030-a02f-13b76e01d234 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:45.623752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7828-224b-4030-a02f-13b76e01d234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:45.625163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8f421-22d8-4149-9c23-47433a3d6308 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:45.634554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd7828-224b-4030-a02f-13b76e01d234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:45.636041 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd7828-224b-4030-a02f-13b76e01d234 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-ccfd7828-224b-4030-a02f-13b76e01d234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:45.653996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:45.654965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:45.656345 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:45.656345 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:45.656345 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:45.656345 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 556/1073] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:45.661099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7828-224b-4030-a02f-13b76e01d234 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:45.710111 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:45.730338 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8f421-22d8-4149-9c23-47433a3d6308 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:45.764630 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2df87-4337-44f7-9b22-d1ecf5627fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:45.765259 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2df87-4337-44f7-9b22-d1ecf5627fcf None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:45.765533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2df87-4337-44f7-9b22-d1ecf5627fcf None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:45.765796 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2df87-4337-44f7-9b22-d1ecf5627fcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:45.766037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7828-224b-4030-a02f-13b76e01d234 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:45.772390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2df87-4337-44f7-9b22-d1ecf5627fcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:45.773832 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2df87-4337-44f7-9b22-d1ecf5627fcf None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-3eb2df87-4337-44f7-9b22-d1ecf5627fcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:45.785410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:45.786361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:45.787443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:45.787443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:45.787443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:45.787443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:45.787443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:45.787443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:45.787443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:45.787443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:45.787443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 518/1074] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:45.792125 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2df87-4337-44f7-9b22-d1ecf5627fcf None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:45.801169 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:45.805070 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628550f-88e7-48c9-818c-9f1c76e54485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:45.805625 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628550f-88e7-48c9-818c-9f1c76e54485 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:45.805894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628550f-88e7-48c9-818c-9f1c76e54485 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:45.806159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628550f-88e7-48c9-818c-9f1c76e54485 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:45.806803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628550f-88e7-48c9-818c-9f1c76e54485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:45.807141 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628550f-88e7-48c9-818c-9f1c76e54485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:45.808553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628550f-88e7-48c9-818c-9f1c76e54485 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:45.821197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 557/1075] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:45.917641 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2df87-4337-44f7-9b22-d1ecf5627fcf None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:45.956732 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e683e17-ab14-4e0f-85d3-4f52d7ff7e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:45.957723 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e683e17-ab14-4e0f-85d3-4f52d7ff7e2c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:45.958192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e683e17-ab14-4e0f-85d3-4f52d7ff7e2c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:45.958465 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e683e17-ab14-4e0f-85d3-4f52d7ff7e2c None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:45.959194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e683e17-ab14-4e0f-85d3-4f52d7ff7e2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:45.959502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e683e17-ab14-4e0f-85d3-4f52d7ff7e2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:45.960940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e683e17-ab14-4e0f-85d3-4f52d7ff7e2c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:45.975376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 519/1076] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:45.981413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628550f-88e7-48c9-818c-9f1c76e54485 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:46.087901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628550f-88e7-48c9-818c-9f1c76e54485 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:46.095878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:46.122376 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc292d5-29d7-41b6-ab4f-8ee9ffa97037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:46.123071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc292d5-29d7-41b6-ab4f-8ee9ffa97037 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:46.123371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc292d5-29d7-41b6-ab4f-8ee9ffa97037 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:46.123638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc292d5-29d7-41b6-ab4f-8ee9ffa97037 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/0bb80229c3234d5f9599b16870c69934/roles/64b48b964be3471a89c5b9cf4fc9b990` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:46.125736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc292d5-29d7-41b6-ab4f-8ee9ffa97037 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=0bb80229c3234d5f9599b16870c69934, role_id=64b48b964be3471a89c5b9cf4fc9b990)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:46.127204 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc292d5-29d7-41b6-ab4f-8ee9ffa97037 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:46.140231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 558/1077] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:37:45 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/0bb80229c3234d5f9599b16870c69934/roles/64b48b964be3471a89c5b9cf4fc9b990 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 11:37:46.368761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56159be-87d4-42d7-8a06-1e7ec1b2d1a6 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:46.369261 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56159be-87d4-42d7-8a06-1e7ec1b2d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:46.369261 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56159be-87d4-42d7-8a06-1e7ec1b2d1a6 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:46.370684 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 520/1078] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:46.375921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f192e-730e-4379-9057-e6cdc36e8d92 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:46.376372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f192e-730e-4379-9057-e6cdc36e8d92 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:46.376467 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f192e-730e-4379-9057-e6cdc36e8d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:46.394777 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-644f192e-730e-4379-9057-e6cdc36e8d92 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:46.395968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-644f192e-730e-4379-9057-e6cdc36e8d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:46.469425 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 559/1079] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:46.734762 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56159be-87d4-42d7-8a06-1e7ec1b2d1a6 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:46.802796 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:46.839744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56159be-87d4-42d7-8a06-1e7ec1b2d1a6 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:46.874237 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b132b6-d1af-48d7-a0bc-9cb1fe59ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:46.874845 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b132b6-d1af-48d7-a0bc-9cb1fe59ee21 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:46.875147 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b132b6-d1af-48d7-a0bc-9cb1fe59ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:46.875409 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b132b6-d1af-48d7-a0bc-9cb1fe59ee21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:46.885544 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b132b6-d1af-48d7-a0bc-9cb1fe59ee21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-15b132b6-d1af-48d7-a0bc-9cb1fe59ee21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:46.890368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:46.891352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:46.892246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 521/1080] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:46.897753 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f192e-730e-4379-9057-e6cdc36e8d92 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:46.992640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:46.999419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f192e-730e-4379-9057-e6cdc36e8d92 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:47.037007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4f45b-063d-4f49-97fc-5232cec147a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:47.037617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f45b-063d-4f49-97fc-5232cec147a9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:47.037888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f45b-063d-4f49-97fc-5232cec147a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:47.038151 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f45b-063d-4f49-97fc-5232cec147a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:47.038801 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4f45b-063d-4f49-97fc-5232cec147a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:47.039098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4f45b-063d-4f49-97fc-5232cec147a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:47.040494 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4f45b-063d-4f49-97fc-5232cec147a9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:47.049361 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 560/1081] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:37:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:47.055660 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b132b6-d1af-48d7-a0bc-9cb1fe59ee21 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:47.160296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b132b6-d1af-48d7-a0bc-9cb1fe59ee21 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:47.188554 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:47.197878 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7ad79-b1d1-46fc-ade6-b0ad3fb4abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:47.198505 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7ad79-b1d1-46fc-ade6-b0ad3fb4abe4 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:47.198775 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7ad79-b1d1-46fc-ade6-b0ad3fb4abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:47.199037 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7ad79-b1d1-46fc-ade6-b0ad3fb4abe4 None admin] PATH_INFO: `/v3/users/bb90dd7aa98642bdae5c2daee7855503` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:47.200189 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7ad79-b1d1-46fc-ade6-b0ad3fb4abe4 None admin] RBAC: Authorizing `identity:get_user(user_id=bb90dd7aa98642bdae5c2daee7855503)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:47.201621 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7ad79-b1d1-46fc-ade6-b0ad3fb4abe4 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:47.203743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 522/1082] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:37:47 2022] GET /identity/v3/users/bb90dd7aa98642bdae5c2daee7855503 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:47.209378 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4f45b-063d-4f49-97fc-5232cec147a9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:47.309050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4f45b-063d-4f49-97fc-5232cec147a9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:47.342832 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04668e-c53a-417a-af4c-fe4426cf47b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:47.343479 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04668e-c53a-417a-af4c-fe4426cf47b9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:47.343746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04668e-c53a-417a-af4c-fe4426cf47b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:47.344010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04668e-c53a-417a-af4c-fe4426cf47b9 None admin] PATH_INFO: `/v3/projects/889330087e064f19b3edbe84bf84e3ac` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:47.345136 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04668e-c53a-417a-af4c-fe4426cf47b9 None admin] RBAC: Authorizing `identity:get_project(project_id=889330087e064f19b3edbe84bf84e3ac)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:47.346591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04668e-c53a-417a-af4c-fe4426cf47b9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:47.348455 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 561/1083] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:37:47 2022] GET /identity/v3/projects/889330087e064f19b3edbe84bf84e3ac => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:47.354101 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7ad79-b1d1-46fc-ade6-b0ad3fb4abe4 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:47.453766 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7ad79-b1d1-46fc-ade6-b0ad3fb4abe4 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:47.490873 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe65d-71eb-423c-a92b-0738aac54cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:47.491545 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe65d-71eb-423c-a92b-0738aac54cd9 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:47.491817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe65d-71eb-423c-a92b-0738aac54cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:47.492081 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe65d-71eb-423c-a92b-0738aac54cd9 None admin] PATH_INFO: `/v3/projects/889330087e064f19b3edbe84bf84e3ac/users/bb90dd7aa98642bdae5c2daee7855503/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:47.494022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe65d-71eb-423c-a92b-0738aac54cd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=889330087e064f19b3edbe84bf84e3ac, user_id=bb90dd7aa98642bdae5c2daee7855503, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:47.495491 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe65d-71eb-423c-a92b-0738aac54cd9 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:47.509594 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 523/1084] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:37:47 2022] PUT /identity/v3/projects/889330087e064f19b3edbe84bf84e3ac/users/bb90dd7aa98642bdae5c2daee7855503/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:37:48.005939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa0b6d-8819-4ff2-9552-3b54f5789592 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:48.006490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa0b6d-8819-4ff2-9552-3b54f5789592 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:48.006490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa0b6d-8819-4ff2-9552-3b54f5789592 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:48.007843 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 562/1085] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:48.013078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a64cb-3d18-44c0-959f-72483f3ce1da None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:48.013362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a64cb-3d18-44c0-959f-72483f3ce1da None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:48.013634 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a64cb-3d18-44c0-959f-72483f3ce1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:48.032122 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-a80a64cb-3d18-44c0-959f-72483f3ce1da None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:48.033294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-a80a64cb-3d18-44c0-959f-72483f3ce1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:48.083707 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:48.102821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 524/1086] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:48.371630 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa0b6d-8819-4ff2-9552-3b54f5789592 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:48.402578 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:48.483049 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa0b6d-8819-4ff2-9552-3b54f5789592 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:48.517885 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ea73-1a9e-4815-8b23-9bbbb19756fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:48.518539 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ea73-1a9e-4815-8b23-9bbbb19756fc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:48.518806 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ea73-1a9e-4815-8b23-9bbbb19756fc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:48.519068 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ea73-1a9e-4815-8b23-9bbbb19756fc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:48.529957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48ea73-1a9e-4815-8b23-9bbbb19756fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-fd48ea73-1a9e-4815-8b23-9bbbb19756fc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:48.534870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:48.535862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:48.536699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 563/1087] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Aug 30 11:37:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:48.542293 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a64cb-3d18-44c0-959f-72483f3ce1da None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:48.645197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a64cb-3d18-44c0-959f-72483f3ce1da None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:48.679217 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995f7f5-4576-4d2e-8705-bf6c661c7003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:48.679809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995f7f5-4576-4d2e-8705-bf6c661c7003 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:48.680073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995f7f5-4576-4d2e-8705-bf6c661c7003 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:48.680334 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995f7f5-4576-4d2e-8705-bf6c661c7003 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:48.680984 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995f7f5-4576-4d2e-8705-bf6c661c7003 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:48.681280 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995f7f5-4576-4d2e-8705-bf6c661c7003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:48.682634 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995f7f5-4576-4d2e-8705-bf6c661c7003 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:48.690714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 525/1088] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Aug 30 11:37:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:48.696368 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ea73-1a9e-4815-8b23-9bbbb19756fc None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:48.781438 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:48.797935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ea73-1a9e-4815-8b23-9bbbb19756fc None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:48.832811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530879e-6c47-463e-9b47-a3b32d35cd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:48.833439 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530879e-6c47-463e-9b47-a3b32d35cd38 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:48.833726 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530879e-6c47-463e-9b47-a3b32d35cd38 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:48.833986 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530879e-6c47-463e-9b47-a3b32d35cd38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:48.844711 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530879e-6c47-463e-9b47-a3b32d35cd38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:48.846172 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530879e-6c47-463e-9b47-a3b32d35cd38 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-e530879e-6c47-463e-9b47-a3b32d35cd38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:48.860371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:48.861517 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:48.862459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:48.862459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:48.862459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:48.862459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 564/1089] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:48.867694 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995f7f5-4576-4d2e-8705-bf6c661c7003 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:48.914854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:48.969661 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995f7f5-4576-4d2e-8705-bf6c661c7003 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:49.004304 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a740d2d-7b61-4388-9f8e-ed01370d012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:49.004902 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a740d2d-7b61-4388-9f8e-ed01370d012c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:49.005179 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a740d2d-7b61-4388-9f8e-ed01370d012c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:49.005449 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a740d2d-7b61-4388-9f8e-ed01370d012c None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:49.006090 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a740d2d-7b61-4388-9f8e-ed01370d012c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:49.006392 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a740d2d-7b61-4388-9f8e-ed01370d012c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:49.007823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a740d2d-7b61-4388-9f8e-ed01370d012c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:49.019673 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 526/1090] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:49.025435 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530879e-6c47-463e-9b47-a3b32d35cd38 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:49.130830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530879e-6c47-463e-9b47-a3b32d35cd38 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:49.165570 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba9276-b712-438a-9ded-3496eefdc7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:49.166111 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba9276-b712-438a-9ded-3496eefdc7ec None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:49.166385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba9276-b712-438a-9ded-3496eefdc7ec None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:49.166652 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba9276-b712-438a-9ded-3496eefdc7ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:49.174624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba9276-b712-438a-9ded-3496eefdc7ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:49.176135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba9276-b712-438a-9ded-3496eefdc7ec None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:49.177993 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-e5ba9276-b712-438a-9ded-3496eefdc7ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:37:49.188398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:49.189349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:49.190372 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 565/1091] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Aug 30 11:37:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:49.195884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a740d2d-7b61-4388-9f8e-ed01370d012c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:49.300155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a740d2d-7b61-4388-9f8e-ed01370d012c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:49.307443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:49.335748 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddc2ba-3640-4f31-bf95-0e09d8fe384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:49.336353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddc2ba-3640-4f31-bf95-0e09d8fe384d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:49.336629 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddc2ba-3640-4f31-bf95-0e09d8fe384d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:49.336901 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddc2ba-3640-4f31-bf95-0e09d8fe384d None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:49.337628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddc2ba-3640-4f31-bf95-0e09d8fe384d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:49.337928 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddc2ba-3640-4f31-bf95-0e09d8fe384d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:49.339383 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddc2ba-3640-4f31-bf95-0e09d8fe384d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:49.357024 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 527/1092] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Aug 30 11:37:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:49.362582 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddc2ba-3640-4f31-bf95-0e09d8fe384d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:49.362743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f466-d8dc-4496-8113-e979b48907cd None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:49.362877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f466-d8dc-4496-8113-e979b48907cd None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:49.363198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f466-d8dc-4496-8113-e979b48907cd None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:49.364527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 566/1093] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:49.369530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c1714-afe3-4783-ae7e-018882f9ded6 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:49.369808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c1714-afe3-4783-ae7e-018882f9ded6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:49.370073 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c1714-afe3-4783-ae7e-018882f9ded6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:49.388901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-bf2c1714-afe3-4783-ae7e-018882f9ded6 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:49.390106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-bf2c1714-afe3-4783-ae7e-018882f9ded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:49.459303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddc2ba-3640-4f31-bf95-0e09d8fe384d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:49.461126 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 567/1094] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:49.492041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e459b-9953-40ef-a931-fb950f47e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:49.492646 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e459b-9953-40ef-a931-fb950f47e19b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:49.492922 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e459b-9953-40ef-a931-fb950f47e19b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:49.493187 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e459b-9953-40ef-a931-fb950f47e19b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:49.522345 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:49.563561 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e459b-9953-40ef-a931-fb950f47e19b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:49.565011 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e459b-9953-40ef-a931-fb950f47e19b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:49.664025 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 528/1095] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:37:49.669942 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c1714-afe3-4783-ae7e-018882f9ded6 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:49.723156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e459b-9953-40ef-a931-fb950f47e19b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:49.774210 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c1714-afe3-4783-ae7e-018882f9ded6 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:49.785114 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:49.801635 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:49.809676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9226ad-f960-42f9-ac3e-8adddcba82fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:49.810303 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9226ad-f960-42f9-ac3e-8adddcba82fe None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:49.810572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9226ad-f960-42f9-ac3e-8adddcba82fe None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:49.810835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9226ad-f960-42f9-ac3e-8adddcba82fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:49.821436 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9226ad-f960-42f9-ac3e-8adddcba82fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:49.822765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e459b-9953-40ef-a931-fb950f47e19b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:49.822958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9226ad-f960-42f9-ac3e-8adddcba82fe None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-de9226ad-f960-42f9-ac3e-8adddcba82fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:37:49.837601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:49.838757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:37:49.840015 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:37:49.840015 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:37:49.840015 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:49.840015 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 568/1096] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:49.844428 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9226ad-f960-42f9-ac3e-8adddcba82fe None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:49.858939 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3776af-4beb-47ad-a2ec-050611059864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:49.859591 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3776af-4beb-47ad-a2ec-050611059864 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:49.859868 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3776af-4beb-47ad-a2ec-050611059864 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:49.860131 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3776af-4beb-47ad-a2ec-050611059864 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:49.870080 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3776af-4beb-47ad-a2ec-050611059864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-2e3776af-4beb-47ad-a2ec-050611059864 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:37:49.874943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:49.875890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:37:49.876914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 529/1097] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:37:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:37:49.881995 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3776af-4beb-47ad-a2ec-050611059864 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:49.946591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9226ad-f960-42f9-ac3e-8adddcba82fe None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:49.977919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3776af-4beb-47ad-a2ec-050611059864 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:49.981529 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cc667-2153-4b86-b89b-aa1588956632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:49.982130 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cc667-2153-4b86-b89b-aa1588956632 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:49.982402 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cc667-2153-4b86-b89b-aa1588956632 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:49.982664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cc667-2153-4b86-b89b-aa1588956632 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:49.983324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79cc667-2153-4b86-b89b-aa1588956632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:49.983623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79cc667-2153-4b86-b89b-aa1588956632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:49.984957 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79cc667-2153-4b86-b89b-aa1588956632 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:49.996617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 569/1098] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:50.001076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:50.002329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cc667-2153-4b86-b89b-aa1588956632 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:50.013004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c229a18-d058-4bb5-b9d0-9dcde0d3308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:50.013661 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c229a18-d058-4bb5-b9d0-9dcde0d3308c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:50.013932 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c229a18-d058-4bb5-b9d0-9dcde0d3308c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:50.014200 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c229a18-d058-4bb5-b9d0-9dcde0d3308c None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:50.014856 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c229a18-d058-4bb5-b9d0-9dcde0d3308c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:50.015151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c229a18-d058-4bb5-b9d0-9dcde0d3308c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:50.016545 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c229a18-d058-4bb5-b9d0-9dcde0d3308c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:50.024984 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 530/1099] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:37:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:50.030597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c229a18-d058-4bb5-b9d0-9dcde0d3308c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:50.044084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:50.103406 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cc667-2153-4b86-b89b-aa1588956632 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:50.127059 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c229a18-d058-4bb5-b9d0-9dcde0d3308c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:50.136249 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d7432-ec21-4988-a2cc-65eafafda145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:50.136888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d7432-ec21-4988-a2cc-65eafafda145 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:50.137161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d7432-ec21-4988-a2cc-65eafafda145 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:50.137429 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d7432-ec21-4988-a2cc-65eafafda145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:50.145370 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d7432-ec21-4988-a2cc-65eafafda145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:50.146829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d7432-ec21-4988-a2cc-65eafafda145 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-2b3d7432-ec21-4988-a2cc-65eafafda145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:37:50.158092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:37:50.159097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:37:50.160277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:37:50.160277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:37:50.160277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:37:50.160277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:37:50.160277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:37:50.160277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:37:50.160277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:37:50.160277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:37:50.160786 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0655a29-f173-4a72-a58d-5296ae73330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:50.160786 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0655a29-f173-4a72-a58d-5296ae73330c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:50.160952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 570/1100] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:37:50.161022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0655a29-f173-4a72-a58d-5296ae73330c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:50.161022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0655a29-f173-4a72-a58d-5296ae73330c None admin] PATH_INFO: `/v3/users/bb90dd7aa98642bdae5c2daee7855503` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:50.161022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0655a29-f173-4a72-a58d-5296ae73330c None admin] RBAC: Authorizing `identity:get_user(user_id=bb90dd7aa98642bdae5c2daee7855503)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:50.162266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0655a29-f173-4a72-a58d-5296ae73330c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:50.163869 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 531/1101] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:37:50 2022] GET /identity/v3/users/bb90dd7aa98642bdae5c2daee7855503 => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:50.165206 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d7432-ec21-4988-a2cc-65eafafda145 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:50.169549 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0655a29-f173-4a72-a58d-5296ae73330c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:50.222753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:50.259657 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:50.262779 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d7432-ec21-4988-a2cc-65eafafda145 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:50.270605 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0655a29-f173-4a72-a58d-5296ae73330c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:50.294865 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1afd7-1144-4ad1-be94-64bb4c53e3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:50.295495 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1afd7-1144-4ad1-be94-64bb4c53e3e1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:50.295764 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1afd7-1144-4ad1-be94-64bb4c53e3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:50.296025 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1afd7-1144-4ad1-be94-64bb4c53e3e1 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:50.296700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1afd7-1144-4ad1-be94-64bb4c53e3e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:50.296992 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1afd7-1144-4ad1-be94-64bb4c53e3e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:50.298417 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1afd7-1144-4ad1-be94-64bb4c53e3e1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:50.303740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8fbd7-9994-4f87-a3cb-548822579f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:50.304335 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8fbd7-9994-4f87-a3cb-548822579f60 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:50.304610 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8fbd7-9994-4f87-a3cb-548822579f60 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:50.304874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8fbd7-9994-4f87-a3cb-548822579f60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:50.310980 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 571/1102] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:37:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:50.316758 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1afd7-1144-4ad1-be94-64bb4c53e3e1 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:50.372190 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8fbd7-9994-4f87-a3cb-548822579f60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:50.373638 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8fbd7-9994-4f87-a3cb-548822579f60 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:50.413920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1afd7-1144-4ad1-be94-64bb4c53e3e1 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:50.426475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:50.451828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb477d-1a1f-4a98-b898-4c82f35ad245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:50.452739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb477d-1a1f-4a98-b898-4c82f35ad245 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:50.453157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb477d-1a1f-4a98-b898-4c82f35ad245 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:50.453741 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb477d-1a1f-4a98-b898-4c82f35ad245 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:50.454453 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb477d-1a1f-4a98-b898-4c82f35ad245 None admin] RBAC: Adding query filter params (role.id=64b48b964be3471a89c5b9cf4fc9b990, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=0bb80229c3234d5f9599b16870c69934) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:50.454753 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb477d-1a1f-4a98-b898-4c82f35ad245 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:50.456306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb477d-1a1f-4a98-b898-4c82f35ad245 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:50.462037 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 572/1103] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:50 2022] GET /identity/v3/role_assignments?user.id=0bb80229c3234d5f9599b16870c69934&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=64b48b964be3471a89c5b9cf4fc9b990 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:37:50.475050 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 532/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 11:37:50.481303 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb477d-1a1f-4a98-b898-4c82f35ad245 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:50.508953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:50.590798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb477d-1a1f-4a98-b898-4c82f35ad245 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:50.628163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc0a9e-29a5-473f-a481-d0a818115b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:50.628815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc0a9e-29a5-473f-a481-d0a818115b26 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:50.629091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc0a9e-29a5-473f-a481-d0a818115b26 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:50.629373 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc0a9e-29a5-473f-a481-d0a818115b26 None admin] PATH_INFO: `/v3/projects/889330087e064f19b3edbe84bf84e3ac` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:50.630519 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc0a9e-29a5-473f-a481-d0a818115b26 None admin] RBAC: Authorizing `identity:get_project(project_id=889330087e064f19b3edbe84bf84e3ac)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:50.632020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc0a9e-29a5-473f-a481-d0a818115b26 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:50.633656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 573/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:37:50 2022] GET /identity/v3/projects/889330087e064f19b3edbe84bf84e3ac => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:50.639987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8fbd7-9994-4f87-a3cb-548822579f60 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:50.738811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8fbd7-9994-4f87-a3cb-548822579f60 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:50.772593 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b54b0-4aad-4d9c-ab88-db057553a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:50.773202 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b54b0-4aad-4d9c-ab88-db057553a0be None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:50.773474 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b54b0-4aad-4d9c-ab88-db057553a0be None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:50.773742 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b54b0-4aad-4d9c-ab88-db057553a0be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:50.774496 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b54b0-4aad-4d9c-ab88-db057553a0be None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=889330087e064f19b3edbe84bf84e3ac, user.id=bb90dd7aa98642bdae5c2daee7855503) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:50.774798 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b54b0-4aad-4d9c-ab88-db057553a0be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:50.776263 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b54b0-4aad-4d9c-ab88-db057553a0be None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:50.782474 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 533/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:50 2022] GET /identity/v3/role_assignments?user.id=bb90dd7aa98642bdae5c2daee7855503&scope.project.id=889330087e064f19b3edbe84bf84e3ac&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:52.678604 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c35d9-b382-420c-9d8a-791d7fc394bb None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:52.679492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c35d9-b382-420c-9d8a-791d7fc394bb None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:52.679492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c35d9-b382-420c-9d8a-791d7fc394bb None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:52.681678 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 574/1107] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:52.687150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da122f56-b9a4-4f5b-b21c-816d48e47f62 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:52.687580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da122f56-b9a4-4f5b-b21c-816d48e47f62 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:52.687679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da122f56-b9a4-4f5b-b21c-816d48e47f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:52.707929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-da122f56-b9a4-4f5b-b21c-816d48e47f62 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:52.709030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-da122f56-b9a4-4f5b-b21c-816d48e47f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:52.722239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:52.777783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 534/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:53.039703 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c35d9-b382-420c-9d8a-791d7fc394bb None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:53.145580 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:53.155822 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c35d9-b382-420c-9d8a-791d7fc394bb None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:53.192717 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d39e1-36c8-4e40-8c81-d2ea5cb2c411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:53.193339 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d39e1-36c8-4e40-8c81-d2ea5cb2c411 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:53.193701 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d39e1-36c8-4e40-8c81-d2ea5cb2c411 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:53.193898 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d39e1-36c8-4e40-8c81-d2ea5cb2c411 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:53.195148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d39e1-36c8-4e40-8c81-d2ea5cb2c411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:53.196629 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d39e1-36c8-4e40-8c81-d2ea5cb2c411 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:53.198196 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 575/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:37:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:53.208189 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da122f56-b9a4-4f5b-b21c-816d48e47f62 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:53.315022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da122f56-b9a4-4f5b-b21c-816d48e47f62 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:53.354741 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1bee1-4894-4474-bb7c-1b4ef73a110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:53.355371 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1bee1-4894-4474-bb7c-1b4ef73a110c None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:53.355645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1bee1-4894-4474-bb7c-1b4ef73a110c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:53.355913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1bee1-4894-4474-bb7c-1b4ef73a110c None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:53.356569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1bee1-4894-4474-bb7c-1b4ef73a110c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:53.358004 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1bee1-4894-4474-bb7c-1b4ef73a110c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:53.381185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 535/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:37:53 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:37:55.285850 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1274b-d08c-4233-ab3e-d0fa9ad498fb None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:55.286433 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1274b-d08c-4233-ab3e-d0fa9ad498fb None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:55.286433 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1274b-d08c-4233-ab3e-d0fa9ad498fb None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:55.288632 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 576/1111] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:55.297429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386415e-18ef-4f82-963a-69c63b00241a None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:55.297821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386415e-18ef-4f82-963a-69c63b00241a None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:55.298025 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386415e-18ef-4f82-963a-69c63b00241a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:55.310671 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:55.325631 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-9386415e-18ef-4f82-963a-69c63b00241a None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:55.326814 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-9386415e-18ef-4f82-963a-69c63b00241a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:55.396071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 536/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:55.658540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1274b-d08c-4233-ab3e-d0fa9ad498fb None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:55.772091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1274b-d08c-4233-ab3e-d0fa9ad498fb None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:55.808082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dec77-f9d0-4d3e-91e1-36edbd1c25bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:55.808716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dec77-f9d0-4d3e-91e1-36edbd1c25bc None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:55.808989 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dec77-f9d0-4d3e-91e1-36edbd1c25bc None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:55.809259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dec77-f9d0-4d3e-91e1-36edbd1c25bc None admin] PATH_INFO: `/v3/roles/05aee5278bd743c4bb8fb605feb806a9` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:55.810778 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74dec77-f9d0-4d3e-91e1-36edbd1c25bc None admin] RBAC: Authorizing `identity:get_role(role_id=05aee5278bd743c4bb8fb605feb806a9)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:55.812189 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74dec77-f9d0-4d3e-91e1-36edbd1c25bc None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:55.813527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 577/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:37:55 2022] GET /identity/v3/roles/05aee5278bd743c4bb8fb605feb806a9 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:55.820306 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386415e-18ef-4f82-963a-69c63b00241a None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:55.931216 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386415e-18ef-4f82-963a-69c63b00241a None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:55.971809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e5fb3-c3fa-4f8a-a821-57732f8606b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:55.972593 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e5fb3-c3fa-4f8a-a821-57732f8606b0 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:55.972871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e5fb3-c3fa-4f8a-a821-57732f8606b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:55.973142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e5fb3-c3fa-4f8a-a821-57732f8606b0 None admin] PATH_INFO: `/v3/users/1963ef526e5e424786d385d10efeea66` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:55.975032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:55.988357 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e5fb3-c3fa-4f8a-a821-57732f8606b0 None admin] RBAC: Authorizing `identity:get_user(user_id=1963ef526e5e424786d385d10efeea66)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:55.989810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e5fb3-c3fa-4f8a-a821-57732f8606b0 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:55.991460 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 537/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:37:55 2022] GET /identity/v3/users/1963ef526e5e424786d385d10efeea66 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:55.997535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dec77-f9d0-4d3e-91e1-36edbd1c25bc None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:56.028581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:56.109540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dec77-f9d0-4d3e-91e1-36edbd1c25bc None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:56.145346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981ff79-f8c4-4e37-b158-d24b1e969dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:56.145979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981ff79-f8c4-4e37-b158-d24b1e969dc5 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:56.146250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981ff79-f8c4-4e37-b158-d24b1e969dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:56.146516 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981ff79-f8c4-4e37-b158-d24b1e969dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:56.215574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981ff79-f8c4-4e37-b158-d24b1e969dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:56.216956 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981ff79-f8c4-4e37-b158-d24b1e969dc5 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:56.244260 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:56.318064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 578/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:37:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:37:56.324328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e5fb3-c3fa-4f8a-a821-57732f8606b0 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:56.433996 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e5fb3-c3fa-4f8a-a821-57732f8606b0 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:56.472782 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1091db-85d4-4f4d-9553-86c6d0ec4ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:56.473332 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1091db-85d4-4f4d-9553-86c6d0ec4ddc None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:56.473612 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1091db-85d4-4f4d-9553-86c6d0ec4ddc None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:56.473879 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1091db-85d4-4f4d-9553-86c6d0ec4ddc None admin] PATH_INFO: `/v3/projects/889330087e064f19b3edbe84bf84e3ac` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:56.475160 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1091db-85d4-4f4d-9553-86c6d0ec4ddc None admin] RBAC: Authorizing `identity:get_project(project_id=889330087e064f19b3edbe84bf84e3ac)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:56.476628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1091db-85d4-4f4d-9553-86c6d0ec4ddc None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:56.478243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 538/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:37:56 2022] GET /identity/v3/projects/889330087e064f19b3edbe84bf84e3ac => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:56.484663 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981ff79-f8c4-4e37-b158-d24b1e969dc5 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:56.588550 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981ff79-f8c4-4e37-b158-d24b1e969dc5 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:56.615062 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:56.623969 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fdca9-83f2-4bd8-a19e-a8b969545602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:56.624568 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fdca9-83f2-4bd8-a19e-a8b969545602 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:56.624838 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fdca9-83f2-4bd8-a19e-a8b969545602 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:56.625088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fdca9-83f2-4bd8-a19e-a8b969545602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:56.625800 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fdca9-83f2-4bd8-a19e-a8b969545602 None admin] RBAC: Adding query filter params (role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=889330087e064f19b3edbe84bf84e3ac, user.id=1963ef526e5e424786d385d10efeea66) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:37:56.626097 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fdca9-83f2-4bd8-a19e-a8b969545602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:56.627573 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fdca9-83f2-4bd8-a19e-a8b969545602 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:56.633130 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 579/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:37:56 2022] GET /identity/v3/role_assignments?user.id=1963ef526e5e424786d385d10efeea66&scope.project.id=889330087e064f19b3edbe84bf84e3ac&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:58.475497 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95c55-1663-407c-a133-9eae7a8b1c04 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:58.476075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95c55-1663-407c-a133-9eae7a8b1c04 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:58.476075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95c55-1663-407c-a133-9eae7a8b1c04 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:58.477385 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 539/1118] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:37:58.483692 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e20ce9-830e-47d1-8390-1a1477dc4eb3 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:58.484428 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e20ce9-830e-47d1-8390-1a1477dc4eb3 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:58.484569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e20ce9-830e-47d1-8390-1a1477dc4eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:58.512535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-e5e20ce9-830e-47d1-8390-1a1477dc4eb3 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:37:58.514433 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-e5e20ce9-830e-47d1-8390-1a1477dc4eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:37:58.590027 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 580/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:37:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:37:58.858506 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95c55-1663-407c-a133-9eae7a8b1c04 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:58.930855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:58.975588 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95c55-1663-407c-a133-9eae7a8b1c04 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:59.009820 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e45db-75d0-4aad-acbc-c41b79ecd54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:59.010469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e45db-75d0-4aad-acbc-c41b79ecd54d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:59.010736 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e45db-75d0-4aad-acbc-c41b79ecd54d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:59.011000 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e45db-75d0-4aad-acbc-c41b79ecd54d None admin] PATH_INFO: `/v3/roles/05aee5278bd743c4bb8fb605feb806a9` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:59.012551 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e45db-75d0-4aad-acbc-c41b79ecd54d None admin] RBAC: Authorizing `identity:get_role(role_id=05aee5278bd743c4bb8fb605feb806a9)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:59.013914 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e45db-75d0-4aad-acbc-c41b79ecd54d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:59.015262 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 540/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:37:58 2022] GET /identity/v3/roles/05aee5278bd743c4bb8fb605feb806a9 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:59.021322 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e20ce9-830e-47d1-8390-1a1477dc4eb3 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:59.127763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e20ce9-830e-47d1-8390-1a1477dc4eb3 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:59.151926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:59.167683 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed022-747c-4442-88e6-d4c467e21c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:59.168316 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed022-747c-4442-88e6-d4c467e21c5a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:59.168588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed022-747c-4442-88e6-d4c467e21c5a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:59.168852 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed022-747c-4442-88e6-d4c467e21c5a None admin] PATH_INFO: `/v3/users/1963ef526e5e424786d385d10efeea66` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:59.170031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed022-747c-4442-88e6-d4c467e21c5a None admin] RBAC: Authorizing `identity:get_user(user_id=1963ef526e5e424786d385d10efeea66)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:59.171515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed022-747c-4442-88e6-d4c467e21c5a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:59.173071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 581/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:37:59 2022] GET /identity/v3/users/1963ef526e5e424786d385d10efeea66 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:59.182608 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e45db-75d0-4aad-acbc-c41b79ecd54d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:59.289467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e45db-75d0-4aad-acbc-c41b79ecd54d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:59.315895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:37:59.330469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5754d87-6465-4de0-9438-4ef58d977797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:59.331426 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5754d87-6465-4de0-9438-4ef58d977797 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:59.331697 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5754d87-6465-4de0-9438-4ef58d977797 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:59.331962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5754d87-6465-4de0-9438-4ef58d977797 None admin] PATH_INFO: `/v3/projects/889330087e064f19b3edbe84bf84e3ac` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:59.333137 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5754d87-6465-4de0-9438-4ef58d977797 None admin] RBAC: Authorizing `identity:get_project(project_id=889330087e064f19b3edbe84bf84e3ac)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:59.334566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5754d87-6465-4de0-9438-4ef58d977797 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:59.336196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 541/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:37:59 2022] GET /identity/v3/projects/889330087e064f19b3edbe84bf84e3ac => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:37:59.342867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed022-747c-4442-88e6-d4c467e21c5a None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:37:59.463937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed022-747c-4442-88e6-d4c467e21c5a None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:37:59.500388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21105e1d-a613-48db-8b4b-de0443c7464b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:37:59.501028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21105e1d-a613-48db-8b4b-de0443c7464b None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:37:59.501300 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21105e1d-a613-48db-8b4b-de0443c7464b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:37:59.501565 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21105e1d-a613-48db-8b4b-de0443c7464b None admin] PATH_INFO: `/v3/projects/889330087e064f19b3edbe84bf84e3ac/users/1963ef526e5e424786d385d10efeea66/roles/05aee5278bd743c4bb8fb605feb806a9` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:37:59.504104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21105e1d-a613-48db-8b4b-de0443c7464b None admin] RBAC: Authorizing `identity:create_grant(project_id=889330087e064f19b3edbe84bf84e3ac, user_id=1963ef526e5e424786d385d10efeea66, role_id=05aee5278bd743c4bb8fb605feb806a9)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:37:59.505532 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21105e1d-a613-48db-8b4b-de0443c7464b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:37:59.521494 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 582/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:37:59 2022] PUT /identity/v3/projects/889330087e064f19b3edbe84bf84e3ac/users/1963ef526e5e424786d385d10efeea66/roles/05aee5278bd743c4bb8fb605feb806a9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:38:01.366679 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e46a8b-da3c-4b7a-81ee-2078d45b6880 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:01.367337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e46a8b-da3c-4b7a-81ee-2078d45b6880 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:01.367337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e46a8b-da3c-4b7a-81ee-2078d45b6880 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:01.368731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 542/1124] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:01.374972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284689d-ce7c-441b-ae84-abeac854c548 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:01.375734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284689d-ce7c-441b-ae84-abeac854c548 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:01.375891 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284689d-ce7c-441b-ae84-abeac854c548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:01.397767 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-c284689d-ce7c-441b-ae84-abeac854c548 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:01.398960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c284689d-ce7c-441b-ae84-abeac854c548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:01.450124 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:01.480951 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 583/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:01.744706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e46a8b-da3c-4b7a-81ee-2078d45b6880 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:01.853461 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e46a8b-da3c-4b7a-81ee-2078d45b6880 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:01.888721 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4649-dfee-40a7-ade5-0048f6873b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:01.889325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4649-dfee-40a7-ade5-0048f6873b1b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:01.889596 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4649-dfee-40a7-ade5-0048f6873b1b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:01.889857 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4649-dfee-40a7-ade5-0048f6873b1b None admin] PATH_INFO: `/v3/roles/05aee5278bd743c4bb8fb605feb806a9` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:01.891379 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df4649-dfee-40a7-ade5-0048f6873b1b None admin] RBAC: Authorizing `identity:get_role(role_id=05aee5278bd743c4bb8fb605feb806a9)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:01.892734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df4649-dfee-40a7-ade5-0048f6873b1b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:01.894084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 543/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:38:01 2022] GET /identity/v3/roles/05aee5278bd743c4bb8fb605feb806a9 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:01.900730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284689d-ce7c-441b-ae84-abeac854c548 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:02.005883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284689d-ce7c-441b-ae84-abeac854c548 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:02.044591 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e13be8-5099-4ff6-a28b-244853d67b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:02.045216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e13be8-5099-4ff6-a28b-244853d67b95 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:02.045487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e13be8-5099-4ff6-a28b-244853d67b95 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:02.045755 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e13be8-5099-4ff6-a28b-244853d67b95 None admin] PATH_INFO: `/v3/users/1963ef526e5e424786d385d10efeea66` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:02.046991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e13be8-5099-4ff6-a28b-244853d67b95 None admin] RBAC: Authorizing `identity:get_user(user_id=1963ef526e5e424786d385d10efeea66)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:02.048458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e13be8-5099-4ff6-a28b-244853d67b95 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:02.050346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 584/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:38:01 2022] GET /identity/v3/users/1963ef526e5e424786d385d10efeea66 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:02.056361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4649-dfee-40a7-ade5-0048f6873b1b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:02.121935 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:02.163924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4649-dfee-40a7-ade5-0048f6873b1b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:02.198337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef47697-01c7-44c1-ba7c-7f63e1a04900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:02.198916 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef47697-01c7-44c1-ba7c-7f63e1a04900 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:02.199213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef47697-01c7-44c1-ba7c-7f63e1a04900 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:02.199478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef47697-01c7-44c1-ba7c-7f63e1a04900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:02.277775 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef47697-01c7-44c1-ba7c-7f63e1a04900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:02.279184 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef47697-01c7-44c1-ba7c-7f63e1a04900 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:02.340207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:02.379469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 544/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:38:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:38:02.385676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e13be8-5099-4ff6-a28b-244853d67b95 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:02.437753 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:02.493187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e13be8-5099-4ff6-a28b-244853d67b95 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:02.540305 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cabf15-0fb6-4951-821f-18b04b7e1881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:02.540866 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cabf15-0fb6-4951-821f-18b04b7e1881 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:02.541142 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cabf15-0fb6-4951-821f-18b04b7e1881 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:02.541409 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cabf15-0fb6-4951-821f-18b04b7e1881 None admin] PATH_INFO: `/v3/projects/889330087e064f19b3edbe84bf84e3ac` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:02.542594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cabf15-0fb6-4951-821f-18b04b7e1881 None admin] RBAC: Authorizing `identity:get_project(project_id=889330087e064f19b3edbe84bf84e3ac)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:02.544079 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cabf15-0fb6-4951-821f-18b04b7e1881 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:02.545717 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 585/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:38:02 2022] GET /identity/v3/projects/889330087e064f19b3edbe84bf84e3ac => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:02.552274 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef47697-01c7-44c1-ba7c-7f63e1a04900 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:02.658786 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef47697-01c7-44c1-ba7c-7f63e1a04900 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:02.697121 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eab978-9d82-41ed-a656-291aa75dc00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:02.697713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eab978-9d82-41ed-a656-291aa75dc00a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:02.697986 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eab978-9d82-41ed-a656-291aa75dc00a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:02.698249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eab978-9d82-41ed-a656-291aa75dc00a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:02.698968 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eab978-9d82-41ed-a656-291aa75dc00a None admin] RBAC: Adding query filter params (role.id=05aee5278bd743c4bb8fb605feb806a9, scope.project.id=889330087e064f19b3edbe84bf84e3ac, user.id=1963ef526e5e424786d385d10efeea66) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:02.699302 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eab978-9d82-41ed-a656-291aa75dc00a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:02.700722 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eab978-9d82-41ed-a656-291aa75dc00a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:02.707850 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 545/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:38:02 2022] GET /identity/v3/role_assignments?user.id=1963ef526e5e424786d385d10efeea66&scope.project.id=889330087e064f19b3edbe84bf84e3ac&role.id=05aee5278bd743c4bb8fb605feb806a9 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:04.581199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6842d-feb6-47d8-9cc9-4341fcb8eb88 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:04.582015 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6842d-feb6-47d8-9cc9-4341fcb8eb88 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:04.582129 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6842d-feb6-47d8-9cc9-4341fcb8eb88 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:04.584384 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 586/1131] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:04.589713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93219ff-8cef-4ed0-b564-305635034e95 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:04.590156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93219ff-8cef-4ed0-b564-305635034e95 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:04.590252 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93219ff-8cef-4ed0-b564-305635034e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:04.610179 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-c93219ff-8cef-4ed0-b564-305635034e95 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:04.611378 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c93219ff-8cef-4ed0-b564-305635034e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:04.634156 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:04.679814 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 546/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:04.940031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6842d-feb6-47d8-9cc9-4341fcb8eb88 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:05.059600 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6842d-feb6-47d8-9cc9-4341fcb8eb88 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:05.077169 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:05.098366 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0bdae-3f0e-4c43-9c97-9e104df3cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:05.098989 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0bdae-3f0e-4c43-9c97-9e104df3cae7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:05.099302 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0bdae-3f0e-4c43-9c97-9e104df3cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:05.099569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0bdae-3f0e-4c43-9c97-9e104df3cae7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:05.100739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0bdae-3f0e-4c43-9c97-9e104df3cae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:05.102186 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0bdae-3f0e-4c43-9c97-9e104df3cae7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:05.103791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 587/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:38:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:05.109906 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93219ff-8cef-4ed0-b564-305635034e95 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:05.224568 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93219ff-8cef-4ed0-b564-305635034e95 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:05.260514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706eafc1-f50b-4436-9af2-1820ccc966e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:05.261112 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706eafc1-f50b-4436-9af2-1820ccc966e6 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:05.261385 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706eafc1-f50b-4436-9af2-1820ccc966e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:05.261657 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706eafc1-f50b-4436-9af2-1820ccc966e6 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:05.262305 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706eafc1-f50b-4436-9af2-1820ccc966e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:05.263781 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706eafc1-f50b-4436-9af2-1820ccc966e6 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:05.297326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 547/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:38:05 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:38:07.177698 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acaf9ee-f62e-4bca-9c0d-95c48112f1b5 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:07.178590 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acaf9ee-f62e-4bca-9c0d-95c48112f1b5 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:07.178590 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acaf9ee-f62e-4bca-9c0d-95c48112f1b5 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:07.180809 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 588/1135] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:07.187069 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e14ed-31a4-4671-ab1e-f01d5e4b0c93 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:07.187877 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e14ed-31a4-4671-ab1e-f01d5e4b0c93 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:07.188031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e14ed-31a4-4671-ab1e-f01d5e4b0c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:07.209131 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-a05e14ed-31a4-4671-ab1e-f01d5e4b0c93 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:07.210311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-a05e14ed-31a4-4671-ab1e-f01d5e4b0c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:07.213150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:07.284717 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 548/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:07.553032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acaf9ee-f62e-4bca-9c0d-95c48112f1b5 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:07.662083 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acaf9ee-f62e-4bca-9c0d-95c48112f1b5 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:07.701974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf06c2-5595-4aee-a11e-266e6688fd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:07.702880 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf06c2-5595-4aee-a11e-266e6688fd28 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:07.703633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf06c2-5595-4aee-a11e-266e6688fd28 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:07.704017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf06c2-5595-4aee-a11e-266e6688fd28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:07.706113 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf06c2-5595-4aee-a11e-266e6688fd28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:07.707606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf06c2-5595-4aee-a11e-266e6688fd28 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:07.709153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 589/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:38:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:07.715843 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e14ed-31a4-4671-ab1e-f01d5e4b0c93 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:07.824422 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e14ed-31a4-4671-ab1e-f01d5e4b0c93 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:07.861555 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007b2f9-3e73-47ee-a700-f4b0cf1d6a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:07.862157 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b2f9-3e73-47ee-a700-f4b0cf1d6a7b None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:07.862427 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b2f9-3e73-47ee-a700-f4b0cf1d6a7b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:07.862687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b2f9-3e73-47ee-a700-f4b0cf1d6a7b None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:07.863351 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007b2f9-3e73-47ee-a700-f4b0cf1d6a7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:07.864761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007b2f9-3e73-47ee-a700-f4b0cf1d6a7b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:07.876073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:07.888843 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 549/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:38:07 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:38:09.805970 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b64077-c504-40db-8f86-c2c1e68077d6 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:09.806830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b64077-c504-40db-8f86-c2c1e68077d6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:09.806830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b64077-c504-40db-8f86-c2c1e68077d6 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:09.808940 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 590/1139] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:09.815093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdde831-9a81-4a80-97fe-3cc7551427da None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:09.815885 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdde831-9a81-4a80-97fe-3cc7551427da None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:09.816023 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdde831-9a81-4a80-97fe-3cc7551427da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:09.840347 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-2bdde831-9a81-4a80-97fe-3cc7551427da None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:09.841535 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-2bdde831-9a81-4a80-97fe-3cc7551427da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:09.911962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 550/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:10.178637 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b64077-c504-40db-8f86-c2c1e68077d6 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:10.245489 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:10.296343 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b64077-c504-40db-8f86-c2c1e68077d6 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:10.336519 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8fd6a-589f-4dd0-a148-9c3223044b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:10.337147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8fd6a-589f-4dd0-a148-9c3223044b82 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:10.337422 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8fd6a-589f-4dd0-a148-9c3223044b82 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:10.337690 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8fd6a-589f-4dd0-a148-9c3223044b82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:10.350951 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8fd6a-589f-4dd0-a148-9c3223044b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-77b8fd6a-589f-4dd0-a148-9c3223044b82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:38:10.357099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:38:10.358336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:38:10.359827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 591/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:38:10.367588 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdde831-9a81-4a80-97fe-3cc7551427da None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:10.476547 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdde831-9a81-4a80-97fe-3cc7551427da None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:10.502852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:10.512955 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801931-24cf-4077-8b73-3657c3a577c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:10.513559 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801931-24cf-4077-8b73-3657c3a577c7 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:10.513842 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801931-24cf-4077-8b73-3657c3a577c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:10.514114 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801931-24cf-4077-8b73-3657c3a577c7 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:10.514771 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73801931-24cf-4077-8b73-3657c3a577c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:10.515068 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73801931-24cf-4077-8b73-3657c3a577c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:10.516477 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73801931-24cf-4077-8b73-3657c3a577c7 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:10.524472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 551/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:10.530451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8fd6a-589f-4dd0-a148-9c3223044b82 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:10.633107 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8fd6a-589f-4dd0-a148-9c3223044b82 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:10.646371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:10.669420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019ef95-c32c-4169-a52e-d07d7b164f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:10.670033 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019ef95-c32c-4169-a52e-d07d7b164f81 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:10.670308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019ef95-c32c-4169-a52e-d07d7b164f81 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:10.670575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019ef95-c32c-4169-a52e-d07d7b164f81 None admin] PATH_INFO: `/v3/users/1e76787cd1024efab4ab45fc99ffd72e` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:10.684501 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019ef95-c32c-4169-a52e-d07d7b164f81 None admin] RBAC: Authorizing `identity:get_user(user_id=1e76787cd1024efab4ab45fc99ffd72e)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:10.685950 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019ef95-c32c-4169-a52e-d07d7b164f81 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:10.687577 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 592/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:38:10 2022] GET /identity/v3/users/1e76787cd1024efab4ab45fc99ffd72e => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:10.693685 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801931-24cf-4077-8b73-3657c3a577c7 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:10.796457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801931-24cf-4077-8b73-3657c3a577c7 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:10.830670 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22107e-59f3-485a-8792-c9375782b0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:10.831362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22107e-59f3-485a-8792-c9375782b0d1 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:10.831594 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22107e-59f3-485a-8792-c9375782b0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:10.831857 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22107e-59f3-485a-8792-c9375782b0d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:10.886837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:10.908360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22107e-59f3-485a-8792-c9375782b0d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:10.910079 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22107e-59f3-485a-8792-c9375782b0d1 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:11.019844 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 552/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:38:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:38:11.026139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019ef95-c32c-4169-a52e-d07d7b164f81 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:11.136341 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019ef95-c32c-4169-a52e-d07d7b164f81 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:11.171092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed9c15-4476-4104-a5b6-a5199a07663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:11.171748 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9c15-4476-4104-a5b6-a5199a07663a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:11.172019 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9c15-4476-4104-a5b6-a5199a07663a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:11.172281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9c15-4476-4104-a5b6-a5199a07663a None admin] PATH_INFO: `/v3/projects/e70bf2ac460942c29e015819c9107a57` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:11.173450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed9c15-4476-4104-a5b6-a5199a07663a None admin] RBAC: Authorizing `identity:get_project(project_id=e70bf2ac460942c29e015819c9107a57)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:11.174893 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed9c15-4476-4104-a5b6-a5199a07663a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:11.176532 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 593/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:38:11 2022] GET /identity/v3/projects/e70bf2ac460942c29e015819c9107a57 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:11.183040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22107e-59f3-485a-8792-c9375782b0d1 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:11.276853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:11.291710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22107e-59f3-485a-8792-c9375782b0d1 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:11.326764 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0295429-9e14-4a81-8a9c-47df708e6d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:11.327375 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0295429-9e14-4a81-8a9c-47df708e6d15 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:11.327648 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0295429-9e14-4a81-8a9c-47df708e6d15 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:11.327911 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0295429-9e14-4a81-8a9c-47df708e6d15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:11.328954 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0295429-9e14-4a81-8a9c-47df708e6d15 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=e70bf2ac460942c29e015819c9107a57, user.id=1e76787cd1024efab4ab45fc99ffd72e) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:11.329256 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0295429-9e14-4a81-8a9c-47df708e6d15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:11.330668 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0295429-9e14-4a81-8a9c-47df708e6d15 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:11.336136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 553/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:38:11 2022] GET /identity/v3/role_assignments?user.id=1e76787cd1024efab4ab45fc99ffd72e&scope.project.id=e70bf2ac460942c29e015819c9107a57&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:13.183516 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca8dfe-6ab9-4e9b-bd34-0bedd23dd8e4 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:13.184402 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca8dfe-6ab9-4e9b-bd34-0bedd23dd8e4 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:13.184402 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca8dfe-6ab9-4e9b-bd34-0bedd23dd8e4 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:13.186539 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 594/1147] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:13.192013 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056fc6b-8130-44b1-ad62-a0abcc812520 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:13.192427 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056fc6b-8130-44b1-ad62-a0abcc812520 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:13.192543 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056fc6b-8130-44b1-ad62-a0abcc812520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:13.213772 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-8056fc6b-8130-44b1-ad62-a0abcc812520 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:13.214945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-8056fc6b-8130-44b1-ad62-a0abcc812520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:13.293007 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 554/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:13.556009 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca8dfe-6ab9-4e9b-bd34-0bedd23dd8e4 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:13.601835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:13.668344 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca8dfe-6ab9-4e9b-bd34-0bedd23dd8e4 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:13.704121 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328285e8-b0a0-4d35-a8c5-c0e9a30ebb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:13.704678 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328285e8-b0a0-4d35-a8c5-c0e9a30ebb4b None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:13.704954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328285e8-b0a0-4d35-a8c5-c0e9a30ebb4b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:13.705218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328285e8-b0a0-4d35-a8c5-c0e9a30ebb4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:13.719571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328285e8-b0a0-4d35-a8c5-c0e9a30ebb4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-328285e8-b0a0-4d35-a8c5-c0e9a30ebb4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:38:13.727530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:38:13.728691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:38:13.729540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 595/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:38:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:38:13.735393 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056fc6b-8130-44b1-ad62-a0abcc812520 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:13.833800 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:13.859271 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056fc6b-8130-44b1-ad62-a0abcc812520 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:13.898669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788f4df-7a9a-4c7e-b46a-4ead65cb56e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:13.899301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788f4df-7a9a-4c7e-b46a-4ead65cb56e3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:13.899574 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788f4df-7a9a-4c7e-b46a-4ead65cb56e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:13.899841 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788f4df-7a9a-4c7e-b46a-4ead65cb56e3 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:13.900505 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788f4df-7a9a-4c7e-b46a-4ead65cb56e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:13.900806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788f4df-7a9a-4c7e-b46a-4ead65cb56e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:13.902178 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788f4df-7a9a-4c7e-b46a-4ead65cb56e3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:13.910822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 555/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:38:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:13.920161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328285e8-b0a0-4d35-a8c5-c0e9a30ebb4b None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:14.028648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:14.032887 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328285e8-b0a0-4d35-a8c5-c0e9a30ebb4b None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:14.068144 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcea27-6784-4150-9365-61e77a83d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:14.068764 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcea27-6784-4150-9365-61e77a83d417 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:14.069005 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcea27-6784-4150-9365-61e77a83d417 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:14.069406 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcea27-6784-4150-9365-61e77a83d417 None admin] PATH_INFO: `/v3/users/1e76787cd1024efab4ab45fc99ffd72e` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:14.070459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcea27-6784-4150-9365-61e77a83d417 None admin] RBAC: Authorizing `identity:get_user(user_id=1e76787cd1024efab4ab45fc99ffd72e)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:14.071921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcea27-6784-4150-9365-61e77a83d417 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:14.073487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 596/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:38:13 2022] GET /identity/v3/users/1e76787cd1024efab4ab45fc99ffd72e => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:14.083361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788f4df-7a9a-4c7e-b46a-4ead65cb56e3 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:14.198372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788f4df-7a9a-4c7e-b46a-4ead65cb56e3 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:14.235339 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290cef6-e2a2-4c89-ae87-3c585731c255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:14.235955 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290cef6-e2a2-4c89-ae87-3c585731c255 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:14.236228 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290cef6-e2a2-4c89-ae87-3c585731c255 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:14.236492 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290cef6-e2a2-4c89-ae87-3c585731c255 None admin] PATH_INFO: `/v3/projects/e70bf2ac460942c29e015819c9107a57` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:14.237620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290cef6-e2a2-4c89-ae87-3c585731c255 None admin] RBAC: Authorizing `identity:get_project(project_id=e70bf2ac460942c29e015819c9107a57)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:14.239041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290cef6-e2a2-4c89-ae87-3c585731c255 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:14.240673 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 556/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:38:14 2022] GET /identity/v3/projects/e70bf2ac460942c29e015819c9107a57 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:14.247176 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcea27-6784-4150-9365-61e77a83d417 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:14.352991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcea27-6784-4150-9365-61e77a83d417 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:14.387461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e90cc3-f527-4444-9d0e-604f70194aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:14.388102 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e90cc3-f527-4444-9d0e-604f70194aea None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:14.388366 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e90cc3-f527-4444-9d0e-604f70194aea None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:14.388629 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e90cc3-f527-4444-9d0e-604f70194aea None admin] PATH_INFO: `/v3/projects/e70bf2ac460942c29e015819c9107a57/users/1e76787cd1024efab4ab45fc99ffd72e/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:14.390726 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e90cc3-f527-4444-9d0e-604f70194aea None admin] RBAC: Authorizing `identity:create_grant(project_id=e70bf2ac460942c29e015819c9107a57, user_id=1e76787cd1024efab4ab45fc99ffd72e, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:14.392180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e90cc3-f527-4444-9d0e-604f70194aea None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:14.411176 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 597/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:38:14 2022] PUT /identity/v3/projects/e70bf2ac460942c29e015819c9107a57/users/1e76787cd1024efab4ab45fc99ffd72e/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:38:16.262529 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd662839-2747-4b78-b2f9-53a79defd707 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:16.263094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd662839-2747-4b78-b2f9-53a79defd707 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:16.263094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd662839-2747-4b78-b2f9-53a79defd707 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:16.264446 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 557/1154] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:16.270813 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515f5f9-258b-4259-ae39-6658c4c113c5 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:16.271542 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515f5f9-258b-4259-ae39-6658c4c113c5 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:16.271691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515f5f9-258b-4259-ae39-6658c4c113c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:16.296034 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-9515f5f9-258b-4259-ae39-6658c4c113c5 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:16.297224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-9515f5f9-258b-4259-ae39-6658c4c113c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:16.320619 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:16.372650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 598/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:16.652950 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd662839-2747-4b78-b2f9-53a79defd707 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:16.677116 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:16.772065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd662839-2747-4b78-b2f9-53a79defd707 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:16.809908 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e700d0-d0d5-4c04-a627-65ea4f89751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:16.810526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e700d0-d0d5-4c04-a627-65ea4f89751d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:16.810800 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e700d0-d0d5-4c04-a627-65ea4f89751d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:16.811062 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e700d0-d0d5-4c04-a627-65ea4f89751d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:16.822396 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e700d0-d0d5-4c04-a627-65ea4f89751d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-31e700d0-d0d5-4c04-a627-65ea4f89751d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:38:16.827226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:38:16.828623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:38:16.829507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 558/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:38:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:38:16.834904 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515f5f9-258b-4259-ae39-6658c4c113c5 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:16.941558 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515f5f9-258b-4259-ae39-6658c4c113c5 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:16.979876 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36bda5-2930-4804-9c13-fb9d9856a71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:16.980479 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36bda5-2930-4804-9c13-fb9d9856a71a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:16.980751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36bda5-2930-4804-9c13-fb9d9856a71a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:16.981017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36bda5-2930-4804-9c13-fb9d9856a71a None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:16.981675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36bda5-2930-4804-9c13-fb9d9856a71a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:16.981974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36bda5-2930-4804-9c13-fb9d9856a71a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:16.983388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36bda5-2930-4804-9c13-fb9d9856a71a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:16.991632 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 599/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:38:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:38:16.997563 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e700d0-d0d5-4c04-a627-65ea4f89751d None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:17.075757 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:17.107709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e700d0-d0d5-4c04-a627-65ea4f89751d None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:17.143459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffef85-6fc9-463a-a81d-b8e50dc668d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:17.144060 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffef85-6fc9-463a-a81d-b8e50dc668d2 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:17.144349 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffef85-6fc9-463a-a81d-b8e50dc668d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:17.144618 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffef85-6fc9-463a-a81d-b8e50dc668d2 None admin] PATH_INFO: `/v3/users/1e76787cd1024efab4ab45fc99ffd72e` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:17.145788 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffef85-6fc9-463a-a81d-b8e50dc668d2 None admin] RBAC: Authorizing `identity:get_user(user_id=1e76787cd1024efab4ab45fc99ffd72e)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:17.147254 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffef85-6fc9-463a-a81d-b8e50dc668d2 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:17.148840 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 559/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:38:16 2022] GET /identity/v3/users/1e76787cd1024efab4ab45fc99ffd72e => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:17.158356 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36bda5-2930-4804-9c13-fb9d9856a71a None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:17.181616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:17.272377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36bda5-2930-4804-9c13-fb9d9856a71a None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:17.309157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537b318-37a2-41e5-a3d2-a2741fcf97a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:17.309749 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537b318-37a2-41e5-a3d2-a2741fcf97a9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:17.310016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537b318-37a2-41e5-a3d2-a2741fcf97a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:17.310277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537b318-37a2-41e5-a3d2-a2741fcf97a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:17.386814 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537b318-37a2-41e5-a3d2-a2741fcf97a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:17.388299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537b318-37a2-41e5-a3d2-a2741fcf97a9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:17.406395 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:17.489804 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 600/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:38:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:38:17.499325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffef85-6fc9-463a-a81d-b8e50dc668d2 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:17.611437 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffef85-6fc9-463a-a81d-b8e50dc668d2 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:17.648301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89e598-24f8-4b93-9a84-4f3856132d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:17.648919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89e598-24f8-4b93-9a84-4f3856132d53 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:17.649191 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89e598-24f8-4b93-9a84-4f3856132d53 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:17.649458 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89e598-24f8-4b93-9a84-4f3856132d53 None admin] PATH_INFO: `/v3/projects/e70bf2ac460942c29e015819c9107a57` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:17.650606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89e598-24f8-4b93-9a84-4f3856132d53 None admin] RBAC: Authorizing `identity:get_project(project_id=e70bf2ac460942c29e015819c9107a57)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:17.652074 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89e598-24f8-4b93-9a84-4f3856132d53 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:17.653686 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 560/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:38:17 2022] GET /identity/v3/projects/e70bf2ac460942c29e015819c9107a57 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:17.662147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537b318-37a2-41e5-a3d2-a2741fcf97a9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:17.767926 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537b318-37a2-41e5-a3d2-a2741fcf97a9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:17.785099 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:17.804906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7fa21-b81d-41ba-ab4d-68d1deb2f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:17.805496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7fa21-b81d-41ba-ab4d-68d1deb2f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:17.805762 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7fa21-b81d-41ba-ab4d-68d1deb2f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:17.806025 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7fa21-b81d-41ba-ab4d-68d1deb2f0b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:17.806739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7fa21-b81d-41ba-ab4d-68d1deb2f0b0 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=e70bf2ac460942c29e015819c9107a57, user.id=1e76787cd1024efab4ab45fc99ffd72e) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:17.807033 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7fa21-b81d-41ba-ab4d-68d1deb2f0b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:17.808508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7fa21-b81d-41ba-ab4d-68d1deb2f0b0 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:17.814810 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 601/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:38:17 2022] GET /identity/v3/role_assignments?user.id=1e76787cd1024efab4ab45fc99ffd72e&scope.project.id=e70bf2ac460942c29e015819c9107a57&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:19.697153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2433b-2322-45e9-94c5-2c32fbd3d580 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:19.697696 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2433b-2322-45e9-94c5-2c32fbd3d580 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:19.697696 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2433b-2322-45e9-94c5-2c32fbd3d580 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:19.698954 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 561/1162] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:19.705554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3268ef-f7d1-46b8-a691-cea61ca33a7a None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:19.706365 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3268ef-f7d1-46b8-a691-cea61ca33a7a None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:19.706498 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3268ef-f7d1-46b8-a691-cea61ca33a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:19.733235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-ed3268ef-f7d1-46b8-a691-cea61ca33a7a None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:19.735086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ed3268ef-f7d1-46b8-a691-cea61ca33a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:19.810711 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 602/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:20.077270 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2433b-2322-45e9-94c5-2c32fbd3d580 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:20.090454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:20.191006 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2433b-2322-45e9-94c5-2c32fbd3d580 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:20.226202 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee28f95-b159-4819-94c5-dd87ee2672dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:20.226793 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee28f95-b159-4819-94c5-dd87ee2672dd None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:20.227064 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee28f95-b159-4819-94c5-dd87ee2672dd None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:20.227357 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee28f95-b159-4819-94c5-dd87ee2672dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:20.304394 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee28f95-b159-4819-94c5-dd87ee2672dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:20.305852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee28f95-b159-4819-94c5-dd87ee2672dd None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:20.315414 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:20.403883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 562/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:38:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:38:20.409984 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3268ef-f7d1-46b8-a691-cea61ca33a7a None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:20.514465 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3268ef-f7d1-46b8-a691-cea61ca33a7a None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:20.527510 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:20.550377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2604fb-76e5-42dd-9cd9-ec8e748f0cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:20.551001 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2604fb-76e5-42dd-9cd9-ec8e748f0cd6 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:20.551310 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2604fb-76e5-42dd-9cd9-ec8e748f0cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:20.551571 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2604fb-76e5-42dd-9cd9-ec8e748f0cd6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:20.558462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2604fb-76e5-42dd-9cd9-ec8e748f0cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:20.559934 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2604fb-76e5-42dd-9cd9-ec8e748f0cd6 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-fc2604fb-76e5-42dd-9cd9-ec8e748f0cd6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:38:20.572221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:38:20.573403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:38:20.574505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 603/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:38:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:38:20.579866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee28f95-b159-4819-94c5-dd87ee2672dd None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:20.681504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee28f95-b159-4819-94c5-dd87ee2672dd None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:20.690759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:20.721908 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78594bfc-8563-4427-8b49-c49b2409237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:20.722504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78594bfc-8563-4427-8b49-c49b2409237d None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:20.722759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78594bfc-8563-4427-8b49-c49b2409237d None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:20.723104 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78594bfc-8563-4427-8b49-c49b2409237d None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:20.723775 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78594bfc-8563-4427-8b49-c49b2409237d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:20.724073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78594bfc-8563-4427-8b49-c49b2409237d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:20.725436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78594bfc-8563-4427-8b49-c49b2409237d None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:20.732622 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 563/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Aug 30 11:38:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:38:22.594817 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e613a8-59d3-467d-b149-1f7fee846801 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:22.595738 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e613a8-59d3-467d-b149-1f7fee846801 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:22.595923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e613a8-59d3-467d-b149-1f7fee846801 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:22.598054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 604/1167] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:22.606974 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7db13-ae2d-4493-8f82-5a72b02a8bff None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:22.607436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7db13-ae2d-4493-8f82-5a72b02a8bff None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:22.607615 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7db13-ae2d-4493-8f82-5a72b02a8bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:22.629306 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-6dd7db13-ae2d-4493-8f82-5a72b02a8bff None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:22.630506 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-6dd7db13-ae2d-4493-8f82-5a72b02a8bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:22.702815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 564/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:22.978109 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e613a8-59d3-467d-b149-1f7fee846801 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:23.093680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e613a8-59d3-467d-b149-1f7fee846801 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:23.128229 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5d48-ac1d-4174-95ec-1154b9aadb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:23.128819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5d48-ac1d-4174-95ec-1154b9aadb78 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:23.129094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5d48-ac1d-4174-95ec-1154b9aadb78 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:23.129350 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5d48-ac1d-4174-95ec-1154b9aadb78 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:23.130016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5d48-ac1d-4174-95ec-1154b9aadb78 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:23.131413 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5d48-ac1d-4174-95ec-1154b9aadb78 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:23.150542 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 605/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Tue Aug 30 11:38:22 2022] POST /identity/v3/domains => generated 267 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:38:25.035063 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feab2b-08ec-4505-962b-13cbe69a4625 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:25.036130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feab2b-08ec-4505-962b-13cbe69a4625 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:25.036130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feab2b-08ec-4505-962b-13cbe69a4625 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:25.038094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 565/1170] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:25.045933 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca393ecb-8754-4fb7-b4f7-975aca136174 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:25.046387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca393ecb-8754-4fb7-b4f7-975aca136174 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:25.046481 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca393ecb-8754-4fb7-b4f7-975aca136174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:25.066249 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-ca393ecb-8754-4fb7-b4f7-975aca136174 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:25.067445 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ca393ecb-8754-4fb7-b4f7-975aca136174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:25.095145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:25.139794 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 606/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:38:25.411633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feab2b-08ec-4505-962b-13cbe69a4625 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:25.535270 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feab2b-08ec-4505-962b-13cbe69a4625 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:25.536810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:25.570532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37c901-677e-4fbd-86f2-4dbae5a812ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:25.571160 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37c901-677e-4fbd-86f2-4dbae5a812ae None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:25.571431 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37c901-677e-4fbd-86f2-4dbae5a812ae None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:25.571692 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37c901-677e-4fbd-86f2-4dbae5a812ae None admin] PATH_INFO: `/v3/domains/a768cf13995040b4955f1d6d4e22e64c` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:25.583439 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37c901-677e-4fbd-86f2-4dbae5a812ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=a768cf13995040b4955f1d6d4e22e64c)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:25.584913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37c901-677e-4fbd-86f2-4dbae5a812ae None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:25.586495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 566/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Tue Aug 30 11:38:25 2022] GET /identity/v3/domains/a768cf13995040b4955f1d6d4e22e64c => generated 267 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:25.593159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca393ecb-8754-4fb7-b4f7-975aca136174 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:25.705735 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca393ecb-8754-4fb7-b4f7-975aca136174 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:25.740552 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75f17c-9849-4cc5-9993-dedc0b83ec12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:25.741153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75f17c-9849-4cc5-9993-dedc0b83ec12 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:25.741424 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75f17c-9849-4cc5-9993-dedc0b83ec12 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:25.741685 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75f17c-9849-4cc5-9993-dedc0b83ec12 None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:25.742317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75f17c-9849-4cc5-9993-dedc0b83ec12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:25.743848 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75f17c-9849-4cc5-9993-dedc0b83ec12 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:25.772680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 607/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:38:25 2022] POST /identity/v3/projects => generated 361 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:38:27.679225 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1592a3-073b-4575-9ef3-7bc2a8f71fce None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:27.680053 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1592a3-073b-4575-9ef3-7bc2a8f71fce None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:27.680053 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1592a3-073b-4575-9ef3-7bc2a8f71fce None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:27.682259 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 567/1174] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:27.691251 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ac631-9851-4a64-84f3-eb4c3847616c None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:27.691662 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ac631-9851-4a64-84f3-eb4c3847616c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:27.691761 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ac631-9851-4a64-84f3-eb4c3847616c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:27.716582 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-1d8ac631-9851-4a64-84f3-eb4c3847616c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:27.717768 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-1d8ac631-9851-4a64-84f3-eb4c3847616c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:27.727146 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:27.796081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 608/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:28.068554 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1592a3-073b-4575-9ef3-7bc2a8f71fce None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:28.194499 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1592a3-073b-4575-9ef3-7bc2a8f71fce None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:28.237875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45390a-fc11-4ea5-ac24-a23675655473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:28.238502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45390a-fc11-4ea5-ac24-a23675655473 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:28.238779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45390a-fc11-4ea5-ac24-a23675655473 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:28.239045 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45390a-fc11-4ea5-ac24-a23675655473 None admin] PATH_INFO: `/v3/domains/a768cf13995040b4955f1d6d4e22e64c` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:28.240333 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45390a-fc11-4ea5-ac24-a23675655473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a768cf13995040b4955f1d6d4e22e64c)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:28.241807 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45390a-fc11-4ea5-ac24-a23675655473 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:28.243422 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 568/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Tue Aug 30 11:38:28 2022] GET /identity/v3/domains/a768cf13995040b4955f1d6d4e22e64c => generated 267 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:28.252430 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ac631-9851-4a64-84f3-eb4c3847616c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:28.356946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ac631-9851-4a64-84f3-eb4c3847616c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:28.398215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf2ee3-e2e4-44ea-af31-772cf5f71ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:28.398846 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf2ee3-e2e4-44ea-af31-772cf5f71ada None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:28.399147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf2ee3-e2e4-44ea-af31-772cf5f71ada None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:28.399411 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf2ee3-e2e4-44ea-af31-772cf5f71ada None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:28.400070 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf2ee3-e2e4-44ea-af31-772cf5f71ada None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:28.401510 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf2ee3-e2e4-44ea-af31-772cf5f71ada None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:28.420482 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:28.427048 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 609/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:38:28 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:38:30.348147 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c6da5-22b4-4ff9-b4d4-46ac64fd79fc None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:30.349087 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c6da5-22b4-4ff9-b4d4-46ac64fd79fc None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:30.349087 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c6da5-22b4-4ff9-b4d4-46ac64fd79fc None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:30.351173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 569/1178] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:30.357446 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b123d-061b-4334-8ae6-5cf9066c9f9c None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:30.358231 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b123d-061b-4334-8ae6-5cf9066c9f9c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:30.358375 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b123d-061b-4334-8ae6-5cf9066c9f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:30.384918 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-893b123d-061b-4334-8ae6-5cf9066c9f9c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:30.386102 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-893b123d-061b-4334-8ae6-5cf9066c9f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:30.456494 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 610/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:30.716070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c6da5-22b4-4ff9-b4d4-46ac64fd79fc None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:30.756849 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:30.833533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c6da5-22b4-4ff9-b4d4-46ac64fd79fc None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:30.868232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812d995-7477-4b7c-a4ed-273316436f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:30.868871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812d995-7477-4b7c-a4ed-273316436f56 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:30.869150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812d995-7477-4b7c-a4ed-273316436f56 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:30.869421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812d995-7477-4b7c-a4ed-273316436f56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:30.880913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812d995-7477-4b7c-a4ed-273316436f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-2812d995-7477-4b7c-a4ed-273316436f56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:38:30.885724 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:38:30.886831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:38:30.887821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 570/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:38:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:38:30.893163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b123d-061b-4334-8ae6-5cf9066c9f9c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:31.007715 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b123d-061b-4334-8ae6-5cf9066c9f9c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:31.036718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:31.043451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce97a12-2c8f-4a98-a444-53b13c95f4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:31.044051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce97a12-2c8f-4a98-a444-53b13c95f4b4 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:31.044317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce97a12-2c8f-4a98-a444-53b13c95f4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:31.044575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce97a12-2c8f-4a98-a444-53b13c95f4b4 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:31.045224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce97a12-2c8f-4a98-a444-53b13c95f4b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:31.045525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce97a12-2c8f-4a98-a444-53b13c95f4b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:31.046885 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce97a12-2c8f-4a98-a444-53b13c95f4b4 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:31.055130 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 611/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:38:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:31.061247 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812d995-7477-4b7c-a4ed-273316436f56 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:31.159600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:31.170962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812d995-7477-4b7c-a4ed-273316436f56 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:31.208478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13529844-3226-40c9-b043-b3892b2a0687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:31.209449 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13529844-3226-40c9-b043-b3892b2a0687 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:31.209874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13529844-3226-40c9-b043-b3892b2a0687 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:31.210194 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13529844-3226-40c9-b043-b3892b2a0687 None admin] PATH_INFO: `/v3/users/ae7c75e162864f33b2ff414a463cfa33` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:31.228779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13529844-3226-40c9-b043-b3892b2a0687 None admin] RBAC: Authorizing `identity:get_user(user_id=ae7c75e162864f33b2ff414a463cfa33)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:31.230224 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13529844-3226-40c9-b043-b3892b2a0687 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:31.231908 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 571/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:38:31 2022] GET /identity/v3/users/ae7c75e162864f33b2ff414a463cfa33 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:31.238196 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce97a12-2c8f-4a98-a444-53b13c95f4b4 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:31.357165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce97a12-2c8f-4a98-a444-53b13c95f4b4 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:31.397334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873a643-00d6-44c1-883e-cff4886fd6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:31.397968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873a643-00d6-44c1-883e-cff4886fd6e2 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:31.398265 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873a643-00d6-44c1-883e-cff4886fd6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:31.398603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873a643-00d6-44c1-883e-cff4886fd6e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:31.463989 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:31.474691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873a643-00d6-44c1-883e-cff4886fd6e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:31.476181 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873a643-00d6-44c1-883e-cff4886fd6e2 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:31.582419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 612/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:38:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:38:31.592070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13529844-3226-40c9-b043-b3892b2a0687 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:31.700198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13529844-3226-40c9-b043-b3892b2a0687 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:31.736014 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1e434-dfd7-4123-bf7a-629e26bb9ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:31.736667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1e434-dfd7-4123-bf7a-629e26bb9ea3 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:31.736938 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1e434-dfd7-4123-bf7a-629e26bb9ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:31.737207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1e434-dfd7-4123-bf7a-629e26bb9ea3 None admin] PATH_INFO: `/v3/projects/ea60da2bf9f64c2c914d70c0f3b0d3e5` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:31.738440 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1e434-dfd7-4123-bf7a-629e26bb9ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=ea60da2bf9f64c2c914d70c0f3b0d3e5)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:31.739897 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1e434-dfd7-4123-bf7a-629e26bb9ea3 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:31.741507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 572/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:38:31 2022] GET /identity/v3/projects/ea60da2bf9f64c2c914d70c0f3b0d3e5 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:31.747970 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873a643-00d6-44c1-883e-cff4886fd6e2 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:31.855860 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:31.871656 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873a643-00d6-44c1-883e-cff4886fd6e2 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:31.910477 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81a49a-108c-4c60-bf68-fa14a76cbf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:31.911068 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81a49a-108c-4c60-bf68-fa14a76cbf25 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:31.911383 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81a49a-108c-4c60-bf68-fa14a76cbf25 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:31.911624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81a49a-108c-4c60-bf68-fa14a76cbf25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:31.912344 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81a49a-108c-4c60-bf68-fa14a76cbf25 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=ea60da2bf9f64c2c914d70c0f3b0d3e5, user.id=ae7c75e162864f33b2ff414a463cfa33) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:31.912645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81a49a-108c-4c60-bf68-fa14a76cbf25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:31.914082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81a49a-108c-4c60-bf68-fa14a76cbf25 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:31.919623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 613/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:38:31 2022] GET /identity/v3/role_assignments?user.id=ae7c75e162864f33b2ff414a463cfa33&scope.project.id=ea60da2bf9f64c2c914d70c0f3b0d3e5&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:33.757039 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92137211-313b-4505-b25c-e750a5cbb449 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:33.757602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92137211-313b-4505-b25c-e750a5cbb449 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:33.757602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92137211-313b-4505-b25c-e750a5cbb449 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:33.758919 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 573/1186] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:33.764379 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37dba0-7064-436c-a196-27618b294131 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:33.764812 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37dba0-7064-436c-a196-27618b294131 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:33.764906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37dba0-7064-436c-a196-27618b294131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:33.785054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-2b37dba0-7064-436c-a196-27618b294131 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:33.786209 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-2b37dba0-7064-436c-a196-27618b294131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:33.855171 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 614/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:34.121778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92137211-313b-4505-b25c-e750a5cbb449 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:34.143094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:34.235779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92137211-313b-4505-b25c-e750a5cbb449 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:34.270616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5bb31-7ee8-4e12-b85c-fec3f8402268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:34.271260 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5bb31-7ee8-4e12-b85c-fec3f8402268 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:34.271528 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5bb31-7ee8-4e12-b85c-fec3f8402268 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:34.271791 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5bb31-7ee8-4e12-b85c-fec3f8402268 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:34.282218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5bb31-7ee8-4e12-b85c-fec3f8402268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-b5d5bb31-7ee8-4e12-b85c-fec3f8402268 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:38:34.287151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:38:34.288150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:38:34.289043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 574/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:38:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:38:34.297484 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37dba0-7064-436c-a196-27618b294131 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:34.385458 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:34.408511 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37dba0-7064-436c-a196-27618b294131 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:34.442376 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493bee6-d408-4d40-ad2e-3cc787d21bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:34.443000 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493bee6-d408-4d40-ad2e-3cc787d21bd3 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:34.443319 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493bee6-d408-4d40-ad2e-3cc787d21bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:34.443585 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493bee6-d408-4d40-ad2e-3cc787d21bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:34.444258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493bee6-d408-4d40-ad2e-3cc787d21bd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:34.444558 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493bee6-d408-4d40-ad2e-3cc787d21bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:34.445947 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493bee6-d408-4d40-ad2e-3cc787d21bd3 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:34.454682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 615/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:38:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:34.464042 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5bb31-7ee8-4e12-b85c-fec3f8402268 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:34.549956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:34.579025 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5bb31-7ee8-4e12-b85c-fec3f8402268 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:34.614613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c894aa-dc8e-474f-90be-e6bc6811ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:34.615253 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c894aa-dc8e-474f-90be-e6bc6811ffab None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:34.615530 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c894aa-dc8e-474f-90be-e6bc6811ffab None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:34.615797 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c894aa-dc8e-474f-90be-e6bc6811ffab None admin] PATH_INFO: `/v3/users/ae7c75e162864f33b2ff414a463cfa33` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:34.617033 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c894aa-dc8e-474f-90be-e6bc6811ffab None admin] RBAC: Authorizing `identity:get_user(user_id=ae7c75e162864f33b2ff414a463cfa33)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:34.618472 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c894aa-dc8e-474f-90be-e6bc6811ffab None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:34.620084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 575/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:38:34 2022] GET /identity/v3/users/ae7c75e162864f33b2ff414a463cfa33 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:34.628695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493bee6-d408-4d40-ad2e-3cc787d21bd3 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:34.732362 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493bee6-d408-4d40-ad2e-3cc787d21bd3 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:34.767257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a958a8-b371-4801-861d-5f116f53d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:34.767934 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a958a8-b371-4801-861d-5f116f53d24a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:34.768191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a958a8-b371-4801-861d-5f116f53d24a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:34.768454 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a958a8-b371-4801-861d-5f116f53d24a None admin] PATH_INFO: `/v3/projects/ea60da2bf9f64c2c914d70c0f3b0d3e5` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:34.769622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a958a8-b371-4801-861d-5f116f53d24a None admin] RBAC: Authorizing `identity:get_project(project_id=ea60da2bf9f64c2c914d70c0f3b0d3e5)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:34.771058 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a958a8-b371-4801-861d-5f116f53d24a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:34.772689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 616/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:38:34 2022] GET /identity/v3/projects/ea60da2bf9f64c2c914d70c0f3b0d3e5 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:34.778754 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c894aa-dc8e-474f-90be-e6bc6811ffab None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:34.883355 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c894aa-dc8e-474f-90be-e6bc6811ffab None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:34.923759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd69df5-3a20-4170-9af1-bad2d646f185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:34.924411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd69df5-3a20-4170-9af1-bad2d646f185 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:34.924681 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd69df5-3a20-4170-9af1-bad2d646f185 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:34.924946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd69df5-3a20-4170-9af1-bad2d646f185 None admin] PATH_INFO: `/v3/projects/ea60da2bf9f64c2c914d70c0f3b0d3e5/users/ae7c75e162864f33b2ff414a463cfa33/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:34.927057 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd69df5-3a20-4170-9af1-bad2d646f185 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea60da2bf9f64c2c914d70c0f3b0d3e5, user_id=ae7c75e162864f33b2ff414a463cfa33, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:34.928602 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd69df5-3a20-4170-9af1-bad2d646f185 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:34.948331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 576/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:38:34 2022] PUT /identity/v3/projects/ea60da2bf9f64c2c914d70c0f3b0d3e5/users/ae7c75e162864f33b2ff414a463cfa33/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:38:36.825425 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ecaa3-3c1a-44e6-8636-d8ebb0a93b12 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:36.826290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ecaa3-3c1a-44e6-8636-d8ebb0a93b12 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:36.826290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ecaa3-3c1a-44e6-8636-d8ebb0a93b12 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:36.828883 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 617/1193] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:38:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:38:36.836127 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6745f85-36f7-4ed9-8e98-95e1a1cddefe None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:36.836845 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6745f85-36f7-4ed9-8e98-95e1a1cddefe None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:36.836983 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6745f85-36f7-4ed9-8e98-95e1a1cddefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:36.848995 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:36.863764 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-c6745f85-36f7-4ed9-8e98-95e1a1cddefe None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:38:36.864970 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c6745f85-36f7-4ed9-8e98-95e1a1cddefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:38:36.944861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 577/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:38:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:38:37.215240 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ecaa3-3c1a-44e6-8636-d8ebb0a93b12 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:37.239778 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:37.338559 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ecaa3-3c1a-44e6-8636-d8ebb0a93b12 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:37.381122 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b37067-3f2e-4c86-8b55-c2aba634ec19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:37.381747 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b37067-3f2e-4c86-8b55-c2aba634ec19 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:37.382020 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b37067-3f2e-4c86-8b55-c2aba634ec19 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:37.382281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b37067-3f2e-4c86-8b55-c2aba634ec19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:37.395389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b37067-3f2e-4c86-8b55-c2aba634ec19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-e1b37067-3f2e-4c86-8b55-c2aba634ec19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:38:37.400100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:38:37.401076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:38:37.401966 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 618/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:38:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:38:37.407963 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6745f85-36f7-4ed9-8e98-95e1a1cddefe None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:37.520122 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6745f85-36f7-4ed9-8e98-95e1a1cddefe None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:37.556503 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52908a93-196e-4e86-84b6-7d1be7f9832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:37.557096 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52908a93-196e-4e86-84b6-7d1be7f9832c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:37.557373 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52908a93-196e-4e86-84b6-7d1be7f9832c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:37.557637 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52908a93-196e-4e86-84b6-7d1be7f9832c None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:37.558301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52908a93-196e-4e86-84b6-7d1be7f9832c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:37.558601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52908a93-196e-4e86-84b6-7d1be7f9832c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:37.559998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52908a93-196e-4e86-84b6-7d1be7f9832c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:37.561357 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:37.569236 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 578/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:38:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:37.576157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b37067-3f2e-4c86-8b55-c2aba634ec19 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:37.656181 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:37.687796 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b37067-3f2e-4c86-8b55-c2aba634ec19 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:37.723392 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036470ba-5566-4c58-98b9-4f865f9eccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:37.724016 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036470ba-5566-4c58-98b9-4f865f9eccc1 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:37.724288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036470ba-5566-4c58-98b9-4f865f9eccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:37.724552 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036470ba-5566-4c58-98b9-4f865f9eccc1 None admin] PATH_INFO: `/v3/users/ae7c75e162864f33b2ff414a463cfa33` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:37.725676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036470ba-5566-4c58-98b9-4f865f9eccc1 None admin] RBAC: Authorizing `identity:get_user(user_id=ae7c75e162864f33b2ff414a463cfa33)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:37.727148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036470ba-5566-4c58-98b9-4f865f9eccc1 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:37.728716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 619/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Aug 30 11:38:37 2022] GET /identity/v3/users/ae7c75e162864f33b2ff414a463cfa33 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:37.735105 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52908a93-196e-4e86-84b6-7d1be7f9832c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:37.838668 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52908a93-196e-4e86-84b6-7d1be7f9832c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:37.872186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53144c50-54bb-4b6c-a6fe-e3423ca627b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:37.872764 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53144c50-54bb-4b6c-a6fe-e3423ca627b1 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:37.873034 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53144c50-54bb-4b6c-a6fe-e3423ca627b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:37.873301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53144c50-54bb-4b6c-a6fe-e3423ca627b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:37.934532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:37.944032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53144c50-54bb-4b6c-a6fe-e3423ca627b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:37.945490 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53144c50-54bb-4b6c-a6fe-e3423ca627b1 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:38.060128 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 579/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:38:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:38:38.066811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036470ba-5566-4c58-98b9-4f865f9eccc1 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:38.178050 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036470ba-5566-4c58-98b9-4f865f9eccc1 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:38.214211 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8db0fd-e380-415f-8e0f-2419870f437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:38.214819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8db0fd-e380-415f-8e0f-2419870f437c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:38.215090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8db0fd-e380-415f-8e0f-2419870f437c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:38.215385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8db0fd-e380-415f-8e0f-2419870f437c None admin] PATH_INFO: `/v3/projects/ea60da2bf9f64c2c914d70c0f3b0d3e5` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:38.216609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8db0fd-e380-415f-8e0f-2419870f437c None admin] RBAC: Authorizing `identity:get_project(project_id=ea60da2bf9f64c2c914d70c0f3b0d3e5)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:38.218038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8db0fd-e380-415f-8e0f-2419870f437c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:38.219655 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 620/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Aug 30 11:38:38 2022] GET /identity/v3/projects/ea60da2bf9f64c2c914d70c0f3b0d3e5 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:38.229195 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53144c50-54bb-4b6c-a6fe-e3423ca627b1 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:38:38.326988 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:38.333626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53144c50-54bb-4b6c-a6fe-e3423ca627b1 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:38:38.369969 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85b9b7-7f70-4691-836a-02121470786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:38:38.370571 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85b9b7-7f70-4691-836a-02121470786c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:38:38.370845 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85b9b7-7f70-4691-836a-02121470786c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:38:38.371110 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85b9b7-7f70-4691-836a-02121470786c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:38:38.371859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85b9b7-7f70-4691-836a-02121470786c None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=ea60da2bf9f64c2c914d70c0f3b0d3e5, user.id=ae7c75e162864f33b2ff414a463cfa33) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:38:38.372165 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85b9b7-7f70-4691-836a-02121470786c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:38:38.373600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85b9b7-7f70-4691-836a-02121470786c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:38:38.379501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 580/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:38:38 2022] GET /identity/v3/role_assignments?user.id=ae7c75e162864f33b2ff414a463cfa33&scope.project.id=ea60da2bf9f64c2c914d70c0f3b0d3e5&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:38:48.208784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=94 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:38:48.376917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=20 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:33.210338 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cbf90-c9af-412b-95e2-bac9212f40e4 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:33.210918 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cbf90-c9af-412b-95e2-bac9212f40e4 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:33.210918 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cbf90-c9af-412b-95e2-bac9212f40e4 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:33.212480 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 621/1201] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:39:33.217862 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93dd3c-943d-4ad7-9d45-1c67dfad0419 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:33.218601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93dd3c-943d-4ad7-9d45-1c67dfad0419 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:33.218601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93dd3c-943d-4ad7-9d45-1c67dfad0419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:33.237459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-ee93dd3c-943d-4ad7-9d45-1c67dfad0419 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:39:33.238611 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-ee93dd3c-943d-4ad7-9d45-1c67dfad0419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:39:33.323702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 581/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:39:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:39:33.638944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cbf90-c9af-412b-95e2-bac9212f40e4 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:33.791081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cbf90-c9af-412b-95e2-bac9212f40e4 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:33.832572 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b7d98-6396-452e-9793-054e09ec8713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:33.833659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b7d98-6396-452e-9793-054e09ec8713 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:33.833659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b7d98-6396-452e-9793-054e09ec8713 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:33.833831 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b7d98-6396-452e-9793-054e09ec8713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:33.849025 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b7d98-6396-452e-9793-054e09ec8713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:33.850865 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b7d98-6396-452e-9793-054e09ec8713 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-4b7b7d98-6396-452e-9793-054e09ec8713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:33.868090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:33.869268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:33.870269 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:33.870269 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:33.870269 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:33.871302 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 622/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:39:33.875694 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93dd3c-943d-4ad7-9d45-1c67dfad0419 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:34.005595 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93dd3c-943d-4ad7-9d45-1c67dfad0419 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:34.043488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:34.050014 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d30db4-8906-4bfb-bff8-d016253b3b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:34.050635 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30db4-8906-4bfb-bff8-d016253b3b69 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:34.050913 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30db4-8906-4bfb-bff8-d016253b3b69 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:34.051288 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30db4-8906-4bfb-bff8-d016253b3b69 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:34.051852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d30db4-8906-4bfb-bff8-d016253b3b69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:34.052153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d30db4-8906-4bfb-bff8-d016253b3b69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:34.053527 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d30db4-8906-4bfb-bff8-d016253b3b69 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:34.067507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 582/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:39:34.074833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b7d98-6396-452e-9793-054e09ec8713 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:34.157722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:34.228618 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b7d98-6396-452e-9793-054e09ec8713 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:34.272014 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c288d8b-d48e-49e7-bc93-301d09719d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:34.272590 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c288d8b-d48e-49e7-bc93-301d09719d29 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:34.272902 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c288d8b-d48e-49e7-bc93-301d09719d29 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:34.273183 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c288d8b-d48e-49e7-bc93-301d09719d29 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:34.273906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c288d8b-d48e-49e7-bc93-301d09719d29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:34.275387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c288d8b-d48e-49e7-bc93-301d09719d29 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:34.299824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 623/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Aug 30 11:39:34 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:39:36.622268 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ae3d2-5084-48a1-8bb2-7aa8a9aa797b None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:36.622816 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ae3d2-5084-48a1-8bb2-7aa8a9aa797b None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:36.622816 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ae3d2-5084-48a1-8bb2-7aa8a9aa797b None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:36.624424 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 583/1206] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:39:36.630610 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de4d66-f980-4138-8b04-a84ce6568a65 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:36.631185 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de4d66-f980-4138-8b04-a84ce6568a65 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:36.631312 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de4d66-f980-4138-8b04-a84ce6568a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:36.653060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-76de4d66-f980-4138-8b04-a84ce6568a65 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:39:36.654692 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-76de4d66-f980-4138-8b04-a84ce6568a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:39:36.740968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 624/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:39:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:39:37.004628 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ae3d2-5084-48a1-8bb2-7aa8a9aa797b None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:37.139594 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ae3d2-5084-48a1-8bb2-7aa8a9aa797b None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:37.182499 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500c69b-21f7-45a9-8dea-c00aee82b404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:37.183240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500c69b-21f7-45a9-8dea-c00aee82b404 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:37.183538 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500c69b-21f7-45a9-8dea-c00aee82b404 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:37.183846 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500c69b-21f7-45a9-8dea-c00aee82b404 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:37.198611 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500c69b-21f7-45a9-8dea-c00aee82b404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-2500c69b-21f7-45a9-8dea-c00aee82b404 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:39:37.204991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:37.206749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:37.207643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 584/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:39:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:39:37.215146 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de4d66-f980-4138-8b04-a84ce6568a65 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:37.263342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:37.354060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de4d66-f980-4138-8b04-a84ce6568a65 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:37.398622 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017e06a-9ece-410b-94f6-268f6ce81e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:37.399487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017e06a-9ece-410b-94f6-268f6ce81e89 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:37.401035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017e06a-9ece-410b-94f6-268f6ce81e89 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:37.401035 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017e06a-9ece-410b-94f6-268f6ce81e89 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:37.401314 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017e06a-9ece-410b-94f6-268f6ce81e89 None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:37.401399 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017e06a-9ece-410b-94f6-268f6ce81e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:37.403282 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017e06a-9ece-410b-94f6-268f6ce81e89 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:37.415168 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 625/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:39:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:39:37.419973 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500c69b-21f7-45a9-8dea-c00aee82b404 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:37.481175 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:37.550899 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500c69b-21f7-45a9-8dea-c00aee82b404 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:37.593072 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd858a3-0035-448c-95b5-a962ce1d44cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:37.593980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd858a3-0035-448c-95b5-a962ce1d44cb None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:37.594391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd858a3-0035-448c-95b5-a962ce1d44cb None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:37.594930 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd858a3-0035-448c-95b5-a962ce1d44cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:37.610429 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd858a3-0035-448c-95b5-a962ce1d44cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:37.612109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd858a3-0035-448c-95b5-a962ce1d44cb None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-1fd858a3-0035-448c-95b5-a962ce1d44cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:37.627915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:37.629524 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:37.630801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:37.630801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:37.630801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:37.630801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 585/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:37.635497 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017e06a-9ece-410b-94f6-268f6ce81e89 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:37.759824 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:37.763392 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017e06a-9ece-410b-94f6-268f6ce81e89 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:37.806877 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b4412-a782-40b9-8725-d75557fd3f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:37.807447 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b4412-a782-40b9-8725-d75557fd3f69 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:37.807723 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b4412-a782-40b9-8725-d75557fd3f69 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:37.808017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b4412-a782-40b9-8725-d75557fd3f69 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:37.808722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b4412-a782-40b9-8725-d75557fd3f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:37.809031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b4412-a782-40b9-8725-d75557fd3f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:37.810589 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b4412-a782-40b9-8725-d75557fd3f69 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:37.825364 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 626/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:37.832041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd858a3-0035-448c-95b5-a962ce1d44cb None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:37.959931 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd858a3-0035-448c-95b5-a962ce1d44cb None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:37.961766 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:38.005428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b198843-1b21-4c5c-81d2-5bc35ae3e5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:38.006136 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b198843-1b21-4c5c-81d2-5bc35ae3e5ff None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:38.006418 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b198843-1b21-4c5c-81d2-5bc35ae3e5ff None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:38.006691 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b198843-1b21-4c5c-81d2-5bc35ae3e5ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:38.015873 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b198843-1b21-4c5c-81d2-5bc35ae3e5ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:38.018965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b198843-1b21-4c5c-81d2-5bc35ae3e5ff None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-6b198843-1b21-4c5c-81d2-5bc35ae3e5ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:39:38.032469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:38.033803 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:38.034999 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 586/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:38.041965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b4412-a782-40b9-8725-d75557fd3f69 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:38.165754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b4412-a782-40b9-8725-d75557fd3f69 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:38.209165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a79f58-33a6-430d-b1a7-fceb60cc378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:38.210090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a79f58-33a6-430d-b1a7-fceb60cc378c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:38.210368 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a79f58-33a6-430d-b1a7-fceb60cc378c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:38.210631 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a79f58-33a6-430d-b1a7-fceb60cc378c None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:38.211415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a79f58-33a6-430d-b1a7-fceb60cc378c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:38.211842 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a79f58-33a6-430d-b1a7-fceb60cc378c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:38.213366 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a79f58-33a6-430d-b1a7-fceb60cc378c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:38.237861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 627/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Aug 30 11:39:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:38.239146 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b198843-1b21-4c5c-81d2-5bc35ae3e5ff None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:38.356510 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b198843-1b21-4c5c-81d2-5bc35ae3e5ff None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:38.398857 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0344c-3662-46ae-992a-3fe2e526dcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:38.399560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0344c-3662-46ae-992a-3fe2e526dcca None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:38.399880 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0344c-3662-46ae-992a-3fe2e526dcca None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:38.400155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0344c-3662-46ae-992a-3fe2e526dcca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:38.446408 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:38.494336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0344c-3662-46ae-992a-3fe2e526dcca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:38.496125 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0344c-3662-46ae-992a-3fe2e526dcca None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:38.615529 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 587/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:39:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:39:38.621596 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a79f58-33a6-430d-b1a7-fceb60cc378c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:38.643110 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:38.729579 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a79f58-33a6-430d-b1a7-fceb60cc378c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:38.764174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d98977-95e7-4a3f-8843-d21e6e155e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:38.764813 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d98977-95e7-4a3f-8843-d21e6e155e35 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:38.765084 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d98977-95e7-4a3f-8843-d21e6e155e35 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:38.765349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d98977-95e7-4a3f-8843-d21e6e155e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:38.776324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d98977-95e7-4a3f-8843-d21e6e155e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:38.777821 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d98977-95e7-4a3f-8843-d21e6e155e35 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-27d98977-95e7-4a3f-8843-d21e6e155e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:38.793333 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:38.801861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:38.803812 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:38.803812 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:38.803812 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:38.803812 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 628/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:38.807168 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0344c-3662-46ae-992a-3fe2e526dcca None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:38.872176 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:38.914450 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0344c-3662-46ae-992a-3fe2e526dcca None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:38.951238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ff18b-e81e-496c-ad9b-d22b0809ee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:38.951845 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ff18b-e81e-496c-ad9b-d22b0809ee3f None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:38.952430 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ff18b-e81e-496c-ad9b-d22b0809ee3f None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:38.952697 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ff18b-e81e-496c-ad9b-d22b0809ee3f None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:38.953397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ff18b-e81e-496c-ad9b-d22b0809ee3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:38.953695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ff18b-e81e-496c-ad9b-d22b0809ee3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:38.955048 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ff18b-e81e-496c-ad9b-d22b0809ee3f None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:38.968675 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 588/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:38.974729 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d98977-95e7-4a3f-8843-d21e6e155e35 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:39.048931 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:39.081841 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d98977-95e7-4a3f-8843-d21e6e155e35 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:39.117782 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e98c4-21cc-4f24-bdb6-9e204e9aae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:39.118592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e98c4-21cc-4f24-bdb6-9e204e9aae73 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:39.118965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e98c4-21cc-4f24-bdb6-9e204e9aae73 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:39.119356 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e98c4-21cc-4f24-bdb6-9e204e9aae73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:39.126096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e98c4-21cc-4f24-bdb6-9e204e9aae73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:39.127674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e98c4-21cc-4f24-bdb6-9e204e9aae73 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-5d6e98c4-21cc-4f24-bdb6-9e204e9aae73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:39:39.138973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:39.140259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:39.141473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:39:39.141473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:39:39.141473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:39:39.141473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:39:39.141473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:39:39.141473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:39:39.141473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:39.141473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:39.147500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 629/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:39.148674 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ff18b-e81e-496c-ad9b-d22b0809ee3f None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:39.257099 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ff18b-e81e-496c-ad9b-d22b0809ee3f None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:39.286234 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:39.305865 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fd31a-8141-4638-87e4-09eababc44ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:39.306549 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fd31a-8141-4638-87e4-09eababc44ec None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:39.306915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fd31a-8141-4638-87e4-09eababc44ec None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:39.307305 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fd31a-8141-4638-87e4-09eababc44ec None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:39.308098 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fd31a-8141-4638-87e4-09eababc44ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:39.308500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fd31a-8141-4638-87e4-09eababc44ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:39.310386 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fd31a-8141-4638-87e4-09eababc44ec None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:39.333214 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 589/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:39.334592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e98c4-21cc-4f24-bdb6-9e204e9aae73 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:39.458203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e98c4-21cc-4f24-bdb6-9e204e9aae73 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:39.487835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:39.495381 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ca651-7403-4f4f-bf6f-2a4f11169e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:39.496014 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ca651-7403-4f4f-bf6f-2a4f11169e68 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:39.496289 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ca651-7403-4f4f-bf6f-2a4f11169e68 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:39.496551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ca651-7403-4f4f-bf6f-2a4f11169e68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:39.497297 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ca651-7403-4f4f-bf6f-2a4f11169e68 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=c8208192d40243e9a81afb254a9326b8) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:39.497595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ca651-7403-4f4f-bf6f-2a4f11169e68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:39.499081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ca651-7403-4f4f-bf6f-2a4f11169e68 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:39.507254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 630/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:39:39 2022] GET /identity/v3/role_assignments?user.id=c8208192d40243e9a81afb254a9326b8&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:39:41.483993 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2c36b-a45c-40a6-b099-c4fddc142278 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:41.484739 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2c36b-a45c-40a6-b099-c4fddc142278 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:41.484739 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2c36b-a45c-40a6-b099-c4fddc142278 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:41.486218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 590/1220] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:39:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:39:41.491595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e7e50-9e51-4901-b08d-12c7d096fef6 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:41.492028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e7e50-9e51-4901-b08d-12c7d096fef6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:41.492294 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e7e50-9e51-4901-b08d-12c7d096fef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:41.518588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-499e7e50-9e51-4901-b08d-12c7d096fef6 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:39:41.520111 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-499e7e50-9e51-4901-b08d-12c7d096fef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:39:41.595819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 631/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:39:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:39:41.865744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2c36b-a45c-40a6-b099-c4fddc142278 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:41.988222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2c36b-a45c-40a6-b099-c4fddc142278 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:42.029087 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322527fe-0a74-4138-a276-d8482bf8ef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:42.029708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322527fe-0a74-4138-a276-d8482bf8ef79 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:42.029956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322527fe-0a74-4138-a276-d8482bf8ef79 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:42.030221 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322527fe-0a74-4138-a276-d8482bf8ef79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:42.040815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322527fe-0a74-4138-a276-d8482bf8ef79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-322527fe-0a74-4138-a276-d8482bf8ef79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:39:42.045569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:42.046677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:42.047801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 591/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:39:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:42.053135 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e7e50-9e51-4901-b08d-12c7d096fef6 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:42.160117 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e7e50-9e51-4901-b08d-12c7d096fef6 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:42.187152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:42.199147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baced90-abc3-426a-a9ee-a7a776ef1644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:42.199938 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baced90-abc3-426a-a9ee-a7a776ef1644 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:42.200203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baced90-abc3-426a-a9ee-a7a776ef1644 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:42.200599 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baced90-abc3-426a-a9ee-a7a776ef1644 None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:42.201315 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baced90-abc3-426a-a9ee-a7a776ef1644 None admin] RBAC: Adding query filter params (name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:42.201691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baced90-abc3-426a-a9ee-a7a776ef1644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:42.203535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baced90-abc3-426a-a9ee-a7a776ef1644 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:42.212265 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 632/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:39:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:42.218646 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322527fe-0a74-4138-a276-d8482bf8ef79 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:42.269059 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:42.331475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322527fe-0a74-4138-a276-d8482bf8ef79 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:42.364823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab90cc4-2136-4c56-bc2a-370fc6e9193e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:42.365445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab90cc4-2136-4c56-bc2a-370fc6e9193e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:42.365748 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab90cc4-2136-4c56-bc2a-370fc6e9193e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:42.366010 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab90cc4-2136-4c56-bc2a-370fc6e9193e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:42.377149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab90cc4-2136-4c56-bc2a-370fc6e9193e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:42.378597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab90cc4-2136-4c56-bc2a-370fc6e9193e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-eab90cc4-2136-4c56-bc2a-370fc6e9193e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:42.393022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:42.394211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:42.395284 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:42.395284 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:42.395284 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:42.395284 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 592/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:42.400672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baced90-abc3-426a-a9ee-a7a776ef1644 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:42.516008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baced90-abc3-426a-a9ee-a7a776ef1644 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:42.552324 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2a7dc-571c-4c76-8915-4837f7438d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:42.552962 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2a7dc-571c-4c76-8915-4837f7438d9f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:42.553275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2a7dc-571c-4c76-8915-4837f7438d9f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:42.553576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2a7dc-571c-4c76-8915-4837f7438d9f None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:42.554296 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2a7dc-571c-4c76-8915-4837f7438d9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:42.554649 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2a7dc-571c-4c76-8915-4837f7438d9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:42.556284 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2a7dc-571c-4c76-8915-4837f7438d9f None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:42.569537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 633/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:42.575542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab90cc4-2136-4c56-bc2a-370fc6e9193e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:42.678876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:42.687665 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab90cc4-2136-4c56-bc2a-370fc6e9193e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:42.728546 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b7566-0baa-4996-9e02-3a05192e6e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:42.729177 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b7566-0baa-4996-9e02-3a05192e6e6a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:42.729447 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b7566-0baa-4996-9e02-3a05192e6e6a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:42.729710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b7566-0baa-4996-9e02-3a05192e6e6a None admin] PATH_INFO: `/v3/users/placement` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:42.739403 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b7566-0baa-4996-9e02-3a05192e6e6a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:42.741540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b7566-0baa-4996-9e02-3a05192e6e6a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-228b7566-0baa-4996-9e02-3a05192e6e6a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:39:42.755980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:42.757163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:42.758106 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 593/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:39:42.763567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2a7dc-571c-4c76-8915-4837f7438d9f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:42.820230 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:42.877144 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2a7dc-571c-4c76-8915-4837f7438d9f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:42.922764 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e856c-78c8-485f-b59c-f41f4b131739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:42.923415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e856c-78c8-485f-b59c-f41f4b131739 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:42.923694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e856c-78c8-485f-b59c-f41f4b131739 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:42.923960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e856c-78c8-485f-b59c-f41f4b131739 None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:42.924757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e856c-78c8-485f-b59c-f41f4b131739 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:42.925114 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e856c-78c8-485f-b59c-f41f4b131739 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:42.926801 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e856c-78c8-485f-b59c-f41f4b131739 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:42.948425 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 634/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Aug 30 11:39:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:42.956134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b7566-0baa-4996-9e02-3a05192e6e6a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:43.075147 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b7566-0baa-4996-9e02-3a05192e6e6a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:43.112391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f09291-9f94-4315-a60b-f6108332ae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:43.113312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f09291-9f94-4315-a60b-f6108332ae64 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:43.113587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f09291-9f94-4315-a60b-f6108332ae64 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:43.113851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f09291-9f94-4315-a60b-f6108332ae64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:43.125364 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f09291-9f94-4315-a60b-f6108332ae64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:43.127190 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f09291-9f94-4315-a60b-f6108332ae64 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-e3f09291-9f94-4315-a60b-f6108332ae64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:43.143141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:43.144550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:43.145672 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:43.145672 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:43.145672 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:43.146577 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 594/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:43.152613 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e856c-78c8-485f-b59c-f41f4b131739 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:43.268283 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e856c-78c8-485f-b59c-f41f4b131739 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:43.271213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:43.307912 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8365d-32b2-4e8f-83a2-d01795c54f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:43.308609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8365d-32b2-4e8f-83a2-d01795c54f6d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:43.309004 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8365d-32b2-4e8f-83a2-d01795c54f6d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:43.309317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8365d-32b2-4e8f-83a2-d01795c54f6d None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:43.309946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8365d-32b2-4e8f-83a2-d01795c54f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:43.310278 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8365d-32b2-4e8f-83a2-d01795c54f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:43.311828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8365d-32b2-4e8f-83a2-d01795c54f6d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:43.324389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 635/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:43.331507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f09291-9f94-4315-a60b-f6108332ae64 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:43.343073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:43.445071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f09291-9f94-4315-a60b-f6108332ae64 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:43.480812 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b19e9-8c81-4a1b-b221-1aad0a5efc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:43.481459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b19e9-8c81-4a1b-b221-1aad0a5efc4e None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:43.481736 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b19e9-8c81-4a1b-b221-1aad0a5efc4e None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:43.482003 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b19e9-8c81-4a1b-b221-1aad0a5efc4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:43.488923 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b19e9-8c81-4a1b-b221-1aad0a5efc4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:43.490401 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b19e9-8c81-4a1b-b221-1aad0a5efc4e None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-f99b19e9-8c81-4a1b-b221-1aad0a5efc4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:39:43.502855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:43.509321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:43.510777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:39:43.510777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:39:43.510777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:39:43.510777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:39:43.510777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:39:43.510777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:39:43.510777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:43.510777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:43.510777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 595/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:39:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:39:43.511530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8365d-32b2-4e8f-83a2-d01795c54f6d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:43.626763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8365d-32b2-4e8f-83a2-d01795c54f6d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:43.662142 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f7e46-848c-4eb0-9eb4-eb2344cc522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:43.662751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f7e46-848c-4eb0-9eb4-eb2344cc522e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:43.663029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f7e46-848c-4eb0-9eb4-eb2344cc522e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:43.663332 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f7e46-848c-4eb0-9eb4-eb2344cc522e None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:43.664019 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f7e46-848c-4eb0-9eb4-eb2344cc522e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:43.664320 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f7e46-848c-4eb0-9eb4-eb2344cc522e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:43.665755 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f7e46-848c-4eb0-9eb4-eb2344cc522e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:43.679589 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 636/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:39:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:43.686483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b19e9-8c81-4a1b-b221-1aad0a5efc4e None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:43.758380 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:43.796447 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b19e9-8c81-4a1b-b221-1aad0a5efc4e None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:43.837703 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9bf92-fe6d-4e0b-8859-81b5000f7050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:43.838701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9bf92-fe6d-4e0b-8859-81b5000f7050 None admin] REQUEST_METHOD: `PUT` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:43.838953 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9bf92-fe6d-4e0b-8859-81b5000f7050 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:43.839303 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9bf92-fe6d-4e0b-8859-81b5000f7050 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/c8208192d40243e9a81afb254a9326b8/roles/0ee8ef3183064d76a7e90b3ae6c5376d` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:43.857220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9bf92-fe6d-4e0b-8859-81b5000f7050 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=c8208192d40243e9a81afb254a9326b8, role_id=0ee8ef3183064d76a7e90b3ae6c5376d)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:43.859100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9bf92-fe6d-4e0b-8859-81b5000f7050 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:43.873285 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 596/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:39:43 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/c8208192d40243e9a81afb254a9326b8/roles/0ee8ef3183064d76a7e90b3ae6c5376d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:39:45.920886 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d44ff-2445-4bfe-9cb8-e4ced68ba5ea None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:45.921465 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d44ff-2445-4bfe-9cb8-e4ced68ba5ea None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:45.921465 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d44ff-2445-4bfe-9cb8-e4ced68ba5ea None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:45.922841 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 637/1233] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:39:45.929192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd107e-db35-47c4-8284-c57aee87f0e0 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:45.929696 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd107e-db35-47c4-8284-c57aee87f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:45.929842 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd107e-db35-47c4-8284-c57aee87f0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:45.951250 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-18bd107e-db35-47c4-8284-c57aee87f0e0 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:39:45.952814 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-18bd107e-db35-47c4-8284-c57aee87f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:39:46.032158 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 597/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:39:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:39:46.329949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d44ff-2445-4bfe-9cb8-e4ced68ba5ea None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:46.358672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:46.456576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d44ff-2445-4bfe-9cb8-e4ced68ba5ea None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:46.506674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa146f58-78d1-48ed-83d2-cde751b4681f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:46.507354 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa146f58-78d1-48ed-83d2-cde751b4681f None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:46.507667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa146f58-78d1-48ed-83d2-cde751b4681f None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:46.507872 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa146f58-78d1-48ed-83d2-cde751b4681f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:46.518280 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa146f58-78d1-48ed-83d2-cde751b4681f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-aa146f58-78d1-48ed-83d2-cde751b4681f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:39:46.523096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:46.524257 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:46.525242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 638/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Aug 30 11:39:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:46.530331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd107e-db35-47c4-8284-c57aee87f0e0 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:46.575876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:46.635334 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd107e-db35-47c4-8284-c57aee87f0e0 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:46.673815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cab68-5263-4f87-bbd6-ce01c81d1081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:46.674479 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cab68-5263-4f87-bbd6-ce01c81d1081 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:46.674758 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cab68-5263-4f87-bbd6-ce01c81d1081 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:46.675046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cab68-5263-4f87-bbd6-ce01c81d1081 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:46.675809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cab68-5263-4f87-bbd6-ce01c81d1081 None admin] RBAC: Adding query filter params (name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:46.676157 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cab68-5263-4f87-bbd6-ce01c81d1081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:46.677592 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cab68-5263-4f87-bbd6-ce01c81d1081 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:46.686978 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 598/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Aug 30 11:39:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:46.692965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa146f58-78d1-48ed-83d2-cde751b4681f None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:46.789700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:46.810038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa146f58-78d1-48ed-83d2-cde751b4681f None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:46.851668 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dff3d-22c5-44d5-9b5a-c1ecb0c28200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:46.852368 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dff3d-22c5-44d5-9b5a-c1ecb0c28200 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:46.852698 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dff3d-22c5-44d5-9b5a-c1ecb0c28200 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:46.853022 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dff3d-22c5-44d5-9b5a-c1ecb0c28200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:46.865490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5dff3d-22c5-44d5-9b5a-c1ecb0c28200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:46.867033 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5dff3d-22c5-44d5-9b5a-c1ecb0c28200 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-ef5dff3d-22c5-44d5-9b5a-c1ecb0c28200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:46.883030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:46.884218 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:46.888564 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:46.888564 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:46.888564 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:46.888564 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 639/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:46.890549 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cab68-5263-4f87-bbd6-ce01c81d1081 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:47.017744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cab68-5263-4f87-bbd6-ce01c81d1081 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:47.028153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:47.057163 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68650d1-efc5-41d3-bad5-bd3252cf86af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:47.057893 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68650d1-efc5-41d3-bad5-bd3252cf86af None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:47.058139 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68650d1-efc5-41d3-bad5-bd3252cf86af None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:47.058474 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68650d1-efc5-41d3-bad5-bd3252cf86af None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:47.059203 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68650d1-efc5-41d3-bad5-bd3252cf86af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:47.059531 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68650d1-efc5-41d3-bad5-bd3252cf86af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:47.060976 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68650d1-efc5-41d3-bad5-bd3252cf86af None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:47.074360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 599/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:47.080241 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dff3d-22c5-44d5-9b5a-c1ecb0c28200 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:47.192705 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dff3d-22c5-44d5-9b5a-c1ecb0c28200 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:47.233719 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc739a15-caa1-461b-a351-b2dfaebc8e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:47.234490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc739a15-caa1-461b-a351-b2dfaebc8e2d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:47.234809 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc739a15-caa1-461b-a351-b2dfaebc8e2d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:47.235157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc739a15-caa1-461b-a351-b2dfaebc8e2d None admin] PATH_INFO: `/v3/users/placement` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:47.238159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:47.244306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc739a15-caa1-461b-a351-b2dfaebc8e2d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:47.246237 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc739a15-caa1-461b-a351-b2dfaebc8e2d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-cc739a15-caa1-461b-a351-b2dfaebc8e2d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:39:47.259098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:47.260377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:47.261734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 640/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:47.266319 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68650d1-efc5-41d3-bad5-bd3252cf86af None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:47.375555 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68650d1-efc5-41d3-bad5-bd3252cf86af None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:47.418587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e5c19-82a3-43c4-8794-c8d0c69ee082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:47.419417 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e5c19-82a3-43c4-8794-c8d0c69ee082 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:47.419798 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e5c19-82a3-43c4-8794-c8d0c69ee082 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:47.420073 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e5c19-82a3-43c4-8794-c8d0c69ee082 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:47.420814 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e5c19-82a3-43c4-8794-c8d0c69ee082 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:47.421121 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e5c19-82a3-43c4-8794-c8d0c69ee082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:47.422856 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e5c19-82a3-43c4-8794-c8d0c69ee082 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:47.444817 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 600/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Aug 30 11:39:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:47.451916 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc739a15-caa1-461b-a351-b2dfaebc8e2d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:47.558601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc739a15-caa1-461b-a351-b2dfaebc8e2d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:47.595820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f7634-cb1f-446b-ae07-a7eb2e46b92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:47.596459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f7634-cb1f-446b-ae07-a7eb2e46b92e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:47.596741 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f7634-cb1f-446b-ae07-a7eb2e46b92e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:47.597032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f7634-cb1f-446b-ae07-a7eb2e46b92e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:47.672066 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:47.679846 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f7634-cb1f-446b-ae07-a7eb2e46b92e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:47.681449 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f7634-cb1f-446b-ae07-a7eb2e46b92e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:47.795932 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 641/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:39:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:39:47.797852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e5c19-82a3-43c4-8794-c8d0c69ee082 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:47.829739 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:47.911311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e5c19-82a3-43c4-8794-c8d0c69ee082 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:47.949678 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780a69c-5884-42fe-89bc-bac064b25adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:47.950384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780a69c-5884-42fe-89bc-bac064b25adf None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:47.950669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780a69c-5884-42fe-89bc-bac064b25adf None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:47.950938 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780a69c-5884-42fe-89bc-bac064b25adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:47.963204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780a69c-5884-42fe-89bc-bac064b25adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:47.964915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780a69c-5884-42fe-89bc-bac064b25adf None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-0780a69c-5884-42fe-89bc-bac064b25adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:47.987257 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:47.988570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:47.989667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:47.989667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:47.989667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:47.989667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 601/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:47.996645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f7634-cb1f-446b-ae07-a7eb2e46b92e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:48.103829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:48.110032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f7634-cb1f-446b-ae07-a7eb2e46b92e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:48.148047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5cc77-60e8-4e61-a8d4-d0c9169e5975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:48.148666 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5cc77-60e8-4e61-a8d4-d0c9169e5975 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:48.148939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5cc77-60e8-4e61-a8d4-d0c9169e5975 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:48.149204 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5cc77-60e8-4e61-a8d4-d0c9169e5975 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:48.149840 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5cc77-60e8-4e61-a8d4-d0c9169e5975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:48.150138 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5cc77-60e8-4e61-a8d4-d0c9169e5975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:48.151553 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5cc77-60e8-4e61-a8d4-d0c9169e5975 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:48.163471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 642/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:48.169636 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780a69c-5884-42fe-89bc-bac064b25adf None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:48.257185 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:48.284120 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780a69c-5884-42fe-89bc-bac064b25adf None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:48.317533 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac52e8-c645-4910-aa97-323c64e387b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:48.318166 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac52e8-c645-4910-aa97-323c64e387b6 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:48.318437 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac52e8-c645-4910-aa97-323c64e387b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:48.318701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac52e8-c645-4910-aa97-323c64e387b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:48.326491 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac52e8-c645-4910-aa97-323c64e387b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:48.328567 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac52e8-c645-4910-aa97-323c64e387b6 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-e3ac52e8-c645-4910-aa97-323c64e387b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:39:48.341542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:48.342734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:48.343896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:39:48.343896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:39:48.343896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:39:48.343896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:39:48.343896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:39:48.343896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:39:48.343896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:48.343896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:48.343896 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 602/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:39:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:39:48.348811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5cc77-60e8-4e61-a8d4-d0c9169e5975 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:48.462073 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5cc77-60e8-4e61-a8d4-d0c9169e5975 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:48.498427 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8905f-e284-42ed-99b8-bf1b4f39869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:48.499368 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8905f-e284-42ed-99b8-bf1b4f39869a None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:48.499934 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8905f-e284-42ed-99b8-bf1b4f39869a None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:48.500420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8905f-e284-42ed-99b8-bf1b4f39869a None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:48.501317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8905f-e284-42ed-99b8-bf1b4f39869a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:48.501667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8905f-e284-42ed-99b8-bf1b4f39869a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:48.503274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8905f-e284-42ed-99b8-bf1b4f39869a None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:48.519945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 643/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:39:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:48.527075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac52e8-c645-4910-aa97-323c64e387b6 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:48.638967 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac52e8-c645-4910-aa97-323c64e387b6 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:48.674732 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e590bfa-6072-476b-9b5f-857bd83e9f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:48.675435 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e590bfa-6072-476b-9b5f-857bd83e9f75 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:48.675751 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e590bfa-6072-476b-9b5f-857bd83e9f75 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:48.676077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e590bfa-6072-476b-9b5f-857bd83e9f75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:48.676899 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e590bfa-6072-476b-9b5f-857bd83e9f75 None admin] RBAC: Adding query filter params (role.id=0ee8ef3183064d76a7e90b3ae6c5376d, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=c8208192d40243e9a81afb254a9326b8) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:48.677245 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e590bfa-6072-476b-9b5f-857bd83e9f75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:48.678821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e590bfa-6072-476b-9b5f-857bd83e9f75 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:48.685498 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 603/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:39:48 2022] GET /identity/v3/role_assignments?user.id=c8208192d40243e9a81afb254a9326b8&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0ee8ef3183064d76a7e90b3ae6c5376d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:50.694177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c77d6-cc0b-4abb-80ea-949bf23d9ac8 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:50.694743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c77d6-cc0b-4abb-80ea-949bf23d9ac8 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:50.694743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c77d6-cc0b-4abb-80ea-949bf23d9ac8 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:50.695945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 644/1247] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:39:50.701508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80883339-9bc6-42df-b650-c3f26b859ca7 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:50.702318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80883339-9bc6-42df-b650-c3f26b859ca7 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:50.702397 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80883339-9bc6-42df-b650-c3f26b859ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:50.708853 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:50.722982 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-80883339-9bc6-42df-b650-c3f26b859ca7 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:39:50.724438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-80883339-9bc6-42df-b650-c3f26b859ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:39:50.798482 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 604/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:39:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:39:51.101699 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c77d6-cc0b-4abb-80ea-949bf23d9ac8 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:51.121944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:51.212723 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c77d6-cc0b-4abb-80ea-949bf23d9ac8 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:51.253626 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233bb59-283d-4610-958f-1af33649a56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:51.254362 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233bb59-283d-4610-958f-1af33649a56d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:51.254659 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233bb59-283d-4610-958f-1af33649a56d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:51.254962 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233bb59-283d-4610-958f-1af33649a56d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:51.267206 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233bb59-283d-4610-958f-1af33649a56d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-0233bb59-283d-4610-958f-1af33649a56d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:39:51.272466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:51.273624 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:51.274746 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 645/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:39:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:51.280733 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80883339-9bc6-42df-b650-c3f26b859ca7 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:51.401147 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80883339-9bc6-42df-b650-c3f26b859ca7 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:51.434892 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:51.442482 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3a28f-0a6d-4916-8ce4-94e237ec08a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:51.443103 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3a28f-0a6d-4916-8ce4-94e237ec08a6 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:51.443410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3a28f-0a6d-4916-8ce4-94e237ec08a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:51.443675 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3a28f-0a6d-4916-8ce4-94e237ec08a6 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:51.444471 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3a28f-0a6d-4916-8ce4-94e237ec08a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:51.444806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3a28f-0a6d-4916-8ce4-94e237ec08a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:51.446502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3a28f-0a6d-4916-8ce4-94e237ec08a6 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:51.458075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 605/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:39:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:39:51.465606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233bb59-283d-4610-958f-1af33649a56d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:51.560039 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:51.590028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233bb59-283d-4610-958f-1af33649a56d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:51.632089 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b83c8-8e0e-44af-b2cd-c9e8829ad32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:51.632788 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b83c8-8e0e-44af-b2cd-c9e8829ad32b None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:51.633059 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b83c8-8e0e-44af-b2cd-c9e8829ad32b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:51.633321 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b83c8-8e0e-44af-b2cd-c9e8829ad32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:51.645284 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b83c8-8e0e-44af-b2cd-c9e8829ad32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:51.646828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b83c8-8e0e-44af-b2cd-c9e8829ad32b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-ca4b83c8-8e0e-44af-b2cd-c9e8829ad32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:51.662290 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:51.663535 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:51.664606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:51.664606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:51.664606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:51.664606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 646/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:51.669955 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3a28f-0a6d-4916-8ce4-94e237ec08a6 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:51.777831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3a28f-0a6d-4916-8ce4-94e237ec08a6 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:51.813478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35abae8-e37a-4caa-841d-af64dd1db003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:51.814099 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35abae8-e37a-4caa-841d-af64dd1db003 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:51.814372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35abae8-e37a-4caa-841d-af64dd1db003 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:51.814686 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35abae8-e37a-4caa-841d-af64dd1db003 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:51.815301 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35abae8-e37a-4caa-841d-af64dd1db003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:51.815597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35abae8-e37a-4caa-841d-af64dd1db003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:51.816958 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35abae8-e37a-4caa-841d-af64dd1db003 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:51.830142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 606/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:51.836133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b83c8-8e0e-44af-b2cd-c9e8829ad32b None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:51.948086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b83c8-8e0e-44af-b2cd-c9e8829ad32b None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:51.980284 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:51.989749 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de298a-1e7d-4c69-a605-1a3cfc160363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:51.990555 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de298a-1e7d-4c69-a605-1a3cfc160363 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:51.990889 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de298a-1e7d-4c69-a605-1a3cfc160363 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:51.991299 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de298a-1e7d-4c69-a605-1a3cfc160363 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:52.004249 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de298a-1e7d-4c69-a605-1a3cfc160363 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:52.006103 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de298a-1e7d-4c69-a605-1a3cfc160363 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-c4de298a-1e7d-4c69-a605-1a3cfc160363 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:39:52.021175 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:52.022640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:52.023802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 647/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:52.028940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35abae8-e37a-4caa-841d-af64dd1db003 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:52.090743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:52.136405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35abae8-e37a-4caa-841d-af64dd1db003 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:52.170824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8d62f-a7ac-4752-80f1-22930a52ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:52.171480 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8d62f-a7ac-4752-80f1-22930a52ab8c None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:52.171749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8d62f-a7ac-4752-80f1-22930a52ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:52.172012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8d62f-a7ac-4752-80f1-22930a52ab8c None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:52.172733 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8d62f-a7ac-4752-80f1-22930a52ab8c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:52.173029 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8d62f-a7ac-4752-80f1-22930a52ab8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:52.174466 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8d62f-a7ac-4752-80f1-22930a52ab8c None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:52.192816 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 607/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Aug 30 11:39:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:52.198807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de298a-1e7d-4c69-a605-1a3cfc160363 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:52.326552 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de298a-1e7d-4c69-a605-1a3cfc160363 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:52.366402 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1c03c-d1cb-4018-bb76-e9ff79a18b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:52.367188 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1c03c-d1cb-4018-bb76-e9ff79a18b02 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:52.367520 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1c03c-d1cb-4018-bb76-e9ff79a18b02 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:52.367740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1c03c-d1cb-4018-bb76-e9ff79a18b02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:52.434201 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:52.448552 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1c03c-d1cb-4018-bb76-e9ff79a18b02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:52.450181 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1c03c-d1cb-4018-bb76-e9ff79a18b02 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:52.560866 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 648/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:39:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:39:52.566751 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8d62f-a7ac-4752-80f1-22930a52ab8c None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:52.676310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8d62f-a7ac-4752-80f1-22930a52ab8c None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:52.690779 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:52.714726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ec006-8b95-4ef6-aa69-6bdea4086854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:52.715320 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec006-8b95-4ef6-aa69-6bdea4086854 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:52.715590 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec006-8b95-4ef6-aa69-6bdea4086854 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:52.715854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec006-8b95-4ef6-aa69-6bdea4086854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:52.726633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ec006-8b95-4ef6-aa69-6bdea4086854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:52.728290 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ec006-8b95-4ef6-aa69-6bdea4086854 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-2f3ec006-8b95-4ef6-aa69-6bdea4086854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:52.745316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:52.746620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:52.747966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:52.747966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:52.747966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:52.747966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 608/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:52.752975 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1c03c-d1cb-4018-bb76-e9ff79a18b02 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:52.846475 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:52.865460 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1c03c-d1cb-4018-bb76-e9ff79a18b02 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:52.904534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f03cbc-7170-4941-bd4f-a51da56cc9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:52.905148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f03cbc-7170-4941-bd4f-a51da56cc9e7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:52.905514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f03cbc-7170-4941-bd4f-a51da56cc9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:52.905880 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f03cbc-7170-4941-bd4f-a51da56cc9e7 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:52.906529 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f03cbc-7170-4941-bd4f-a51da56cc9e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:52.906825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f03cbc-7170-4941-bd4f-a51da56cc9e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:52.908278 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f03cbc-7170-4941-bd4f-a51da56cc9e7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:52.923417 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 649/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:52.929414 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ec006-8b95-4ef6-aa69-6bdea4086854 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:53.046292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ec006-8b95-4ef6-aa69-6bdea4086854 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:53.082005 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f250e-d219-441e-b890-f8baf788bb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:53.082839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f250e-d219-441e-b890-f8baf788bb8a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:53.083222 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f250e-d219-441e-b890-f8baf788bb8a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:53.083518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f250e-d219-441e-b890-f8baf788bb8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:53.090948 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f250e-d219-441e-b890-f8baf788bb8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:53.092519 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f250e-d219-441e-b890-f8baf788bb8a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-aa3f250e-d219-441e-b890-f8baf788bb8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:39:53.104352 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:53.105483 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:53.106404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:39:53.106404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:39:53.106404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:39:53.106404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:39:53.106404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:39:53.106404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:39:53.106404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:53.106404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:39:53.106404 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 609/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:39:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:39:53.111958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f03cbc-7170-4941-bd4f-a51da56cc9e7 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:53.213450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f03cbc-7170-4941-bd4f-a51da56cc9e7 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:53.248368 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7569b29-6f34-4484-8f86-4aa36d1728aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:53.248981 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7569b29-6f34-4484-8f86-4aa36d1728aa None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:53.249251 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7569b29-6f34-4484-8f86-4aa36d1728aa None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:53.249566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7569b29-6f34-4484-8f86-4aa36d1728aa None admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:53.250221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7569b29-6f34-4484-8f86-4aa36d1728aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:53.250515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7569b29-6f34-4484-8f86-4aa36d1728aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:53.252109 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7569b29-6f34-4484-8f86-4aa36d1728aa None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:53.264276 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 650/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:39:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:53.270503 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f250e-d219-441e-b890-f8baf788bb8a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:53.303614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:53.380255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f250e-d219-441e-b890-f8baf788bb8a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:53.414866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce91f6-e738-46d9-aa97-ad74b0f0cc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:53.415531 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce91f6-e738-46d9-aa97-ad74b0f0cc26 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:53.415800 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce91f6-e738-46d9-aa97-ad74b0f0cc26 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:53.416115 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce91f6-e738-46d9-aa97-ad74b0f0cc26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:53.416766 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce91f6-e738-46d9-aa97-ad74b0f0cc26 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=c8208192d40243e9a81afb254a9326b8) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:53.417063 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce91f6-e738-46d9-aa97-ad74b0f0cc26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:53.418488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce91f6-e738-46d9-aa97-ad74b0f0cc26 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:53.424135 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 610/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:39:53 2022] GET /identity/v3/role_assignments?user.id=c8208192d40243e9a81afb254a9326b8&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:55.262712 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413c843-bc2b-4f67-aee9-7984e7a3c6e0 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:55.263351 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413c843-bc2b-4f67-aee9-7984e7a3c6e0 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:55.263351 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413c843-bc2b-4f67-aee9-7984e7a3c6e0 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:55.264603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 651/1261] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:39:55.270778 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d42a3-7d90-46a2-a5d3-d4308f96efef None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:55.271575 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d42a3-7d90-46a2-a5d3-d4308f96efef None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:55.271721 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d42a3-7d90-46a2-a5d3-d4308f96efef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:55.294552 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-9f2d42a3-7d90-46a2-a5d3-d4308f96efef None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:39:55.295750 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-9f2d42a3-7d90-46a2-a5d3-d4308f96efef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:39:55.371540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 611/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:39:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:39:55.631657 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413c843-bc2b-4f67-aee9-7984e7a3c6e0 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:55.644155 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:55.741505 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413c843-bc2b-4f67-aee9-7984e7a3c6e0 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:55.776827 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688e35c-ba14-4c42-995f-62c0041f6224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:55.777442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688e35c-ba14-4c42-995f-62c0041f6224 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:55.777717 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688e35c-ba14-4c42-995f-62c0041f6224 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:55.777974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688e35c-ba14-4c42-995f-62c0041f6224 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:55.788876 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688e35c-ba14-4c42-995f-62c0041f6224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-d688e35c-ba14-4c42-995f-62c0041f6224 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:39:55.793745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:55.794826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:55.796060 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 652/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:39:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:55.801405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d42a3-7d90-46a2-a5d3-d4308f96efef None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:55.820726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:55.904040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d42a3-7d90-46a2-a5d3-d4308f96efef None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:55.938451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ae86-56e3-41b7-9981-f8ce41372a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:55.939246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ae86-56e3-41b7-9981-f8ce41372a71 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:55.939526 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ae86-56e3-41b7-9981-f8ce41372a71 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:55.939785 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ae86-56e3-41b7-9981-f8ce41372a71 None admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:55.940445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62ae86-56e3-41b7-9981-f8ce41372a71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:55.940738 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62ae86-56e3-41b7-9981-f8ce41372a71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:55.942093 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62ae86-56e3-41b7-9981-f8ce41372a71 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:55.950075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 612/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:55.956991 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688e35c-ba14-4c42-995f-62c0041f6224 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:56.044508 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:56.084782 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688e35c-ba14-4c42-995f-62c0041f6224 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:56.122698 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251404-b868-4fff-9f99-2325a5927b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:56.123361 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251404-b868-4fff-9f99-2325a5927b66 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:56.123637 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251404-b868-4fff-9f99-2325a5927b66 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:56.123905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251404-b868-4fff-9f99-2325a5927b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:56.134385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea251404-b868-4fff-9f99-2325a5927b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:56.135866 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea251404-b868-4fff-9f99-2325a5927b66 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-ea251404-b868-4fff-9f99-2325a5927b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:56.150775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:56.151907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:56.153056 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:56.153056 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:56.153056 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:56.153056 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 653/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:56.158076 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ae86-56e3-41b7-9981-f8ce41372a71 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:56.251278 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:56.277263 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ae86-56e3-41b7-9981-f8ce41372a71 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:56.313317 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4c643-71bc-4311-8f9b-697ab246590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:56.313910 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4c643-71bc-4311-8f9b-697ab246590b None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:56.314186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4c643-71bc-4311-8f9b-697ab246590b None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:56.314457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4c643-71bc-4311-8f9b-697ab246590b None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:56.315098 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4c643-71bc-4311-8f9b-697ab246590b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:56.315438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4c643-71bc-4311-8f9b-697ab246590b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:56.316801 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4c643-71bc-4311-8f9b-697ab246590b None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:56.328890 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 613/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:56.334766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251404-b868-4fff-9f99-2325a5927b66 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:56.446073 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251404-b868-4fff-9f99-2325a5927b66 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:56.466791 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:56.483700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddba697-3a26-4bd0-b8da-72659233ca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:56.484400 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddba697-3a26-4bd0-b8da-72659233ca8e None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:56.484675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddba697-3a26-4bd0-b8da-72659233ca8e None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:56.484939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddba697-3a26-4bd0-b8da-72659233ca8e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:56.492737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddba697-3a26-4bd0-b8da-72659233ca8e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:56.494182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddba697-3a26-4bd0-b8da-72659233ca8e None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-fddba697-3a26-4bd0-b8da-72659233ca8e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:39:56.505569 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:56.511667 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:56.512763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 654/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:56.513500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4c643-71bc-4311-8f9b-697ab246590b None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:56.619338 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4c643-71bc-4311-8f9b-697ab246590b None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:56.658122 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18573d-ebdd-43e3-86d4-acf03c339041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:56.658720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18573d-ebdd-43e3-86d4-acf03c339041 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:56.658998 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18573d-ebdd-43e3-86d4-acf03c339041 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:56.659298 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18573d-ebdd-43e3-86d4-acf03c339041 None admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:56.660030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18573d-ebdd-43e3-86d4-acf03c339041 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:56.660334 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18573d-ebdd-43e3-86d4-acf03c339041 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:56.661770 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18573d-ebdd-43e3-86d4-acf03c339041 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:56.671019 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:56.680015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 614/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Aug 30 11:39:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:56.688818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddba697-3a26-4bd0-b8da-72659233ca8e None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:56.806680 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddba697-3a26-4bd0-b8da-72659233ca8e None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:56.842303 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596c530-fa00-4591-b7f5-da78d8d0dd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:56.842925 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596c530-fa00-4591-b7f5-da78d8d0dd5d None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:56.843270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596c530-fa00-4591-b7f5-da78d8d0dd5d None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:56.843538 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596c530-fa00-4591-b7f5-da78d8d0dd5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:56.854228 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596c530-fa00-4591-b7f5-da78d8d0dd5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:56.855722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596c530-fa00-4591-b7f5-da78d8d0dd5d None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-6596c530-fa00-4591-b7f5-da78d8d0dd5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:39:56.869954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:56.871063 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:39:56.872398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:39:56.872398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:39:56.872398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:56.872398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 655/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:39:56.877903 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18573d-ebdd-43e3-86d4-acf03c339041 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:56.996478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18573d-ebdd-43e3-86d4-acf03c339041 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:57.034043 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba429fca-b455-484b-a438-ee90fa222cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:57.034645 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba429fca-b455-484b-a438-ee90fa222cc6 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:57.034922 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba429fca-b455-484b-a438-ee90fa222cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:57.035218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba429fca-b455-484b-a438-ee90fa222cc6 None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:57.035866 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba429fca-b455-484b-a438-ee90fa222cc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:57.036149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba429fca-b455-484b-a438-ee90fa222cc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:57.037513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba429fca-b455-484b-a438-ee90fa222cc6 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:57.049001 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 615/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:57.058389 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596c530-fa00-4591-b7f5-da78d8d0dd5d None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:57.097783 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:57.169743 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596c530-fa00-4591-b7f5-da78d8d0dd5d None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:57.205110 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a721-0b7a-4a73-98eb-06d307d6fccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:57.205720 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a721-0b7a-4a73-98eb-06d307d6fccb None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:57.205998 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a721-0b7a-4a73-98eb-06d307d6fccb None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:57.206262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a721-0b7a-4a73-98eb-06d307d6fccb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:57.213251 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54a721-0b7a-4a73-98eb-06d307d6fccb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:57.214689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54a721-0b7a-4a73-98eb-06d307d6fccb None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-be54a721-0b7a-4a73-98eb-06d307d6fccb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:39:57.225892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:39:57.226895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:39:57.228325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:39:57.228325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:39:57.228325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:39:57.228325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:39:57.228325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:39:57.228325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:39:57.228325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:39:57.228325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:39:57.228325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 656/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:39:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:39:57.235501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba429fca-b455-484b-a438-ee90fa222cc6 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:57.296585 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:57.345228 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba429fca-b455-484b-a438-ee90fa222cc6 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:57.391566 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39604217-73fa-4149-81d9-8f7fa5c9d633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:57.392316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39604217-73fa-4149-81d9-8f7fa5c9d633 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:57.392609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39604217-73fa-4149-81d9-8f7fa5c9d633 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:57.392901 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39604217-73fa-4149-81d9-8f7fa5c9d633 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:57.393947 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39604217-73fa-4149-81d9-8f7fa5c9d633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:39:57.394362 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39604217-73fa-4149-81d9-8f7fa5c9d633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:57.396657 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39604217-73fa-4149-81d9-8f7fa5c9d633 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:57.412784 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 616/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:39:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:39:57.419028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a721-0b7a-4a73-98eb-06d307d6fccb None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:57.513428 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:57.522953 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a721-0b7a-4a73-98eb-06d307d6fccb None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:57.558334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445a4ea-e4db-47fb-8138-b4f460e95943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:57.558975 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a4ea-e4db-47fb-8138-b4f460e95943 None admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:57.559274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a4ea-e4db-47fb-8138-b4f460e95943 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:57.559536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a4ea-e4db-47fb-8138-b4f460e95943 None admin] PATH_INFO: `/v3/projects/c0027f77ae674bc08f99838ef869845c/users/c8208192d40243e9a81afb254a9326b8/roles/0209018e60254b91b5b7ea8d0bc5d501` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:57.561583 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445a4ea-e4db-47fb-8138-b4f460e95943 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0027f77ae674bc08f99838ef869845c, user_id=c8208192d40243e9a81afb254a9326b8, role_id=0209018e60254b91b5b7ea8d0bc5d501)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:39:57.563068 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445a4ea-e4db-47fb-8138-b4f460e95943 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:39:57.584070 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 657/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Aug 30 11:39:57 2022] PUT /identity/v3/projects/c0027f77ae674bc08f99838ef869845c/users/c8208192d40243e9a81afb254a9326b8/roles/0209018e60254b91b5b7ea8d0bc5d501 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:39:59.449713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78249f5e-3716-45a9-b022-f289deb80d0e None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:59.450469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78249f5e-3716-45a9-b022-f289deb80d0e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:59.450621 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78249f5e-3716-45a9-b022-f289deb80d0e None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:59.453045 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 617/1274] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:39:59.458263 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd695e-ef56-4713-8798-46a561ec4bf2 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:59.458682 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd695e-ef56-4713-8798-46a561ec4bf2 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:59.458774 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd695e-ef56-4713-8798-46a561ec4bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:59.478615 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-20fd695e-ef56-4713-8798-46a561ec4bf2 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:39:59.479797 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-20fd695e-ef56-4713-8798-46a561ec4bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:39:59.550595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 658/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:39:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:39:59.827692 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78249f5e-3716-45a9-b022-f289deb80d0e None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:39:59.939084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78249f5e-3716-45a9-b022-f289deb80d0e None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:39:59.958464 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:39:59.987072 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f48a6-e214-44c5-8563-0593967ef140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:39:59.987745 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f48a6-e214-44c5-8563-0593967ef140 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:39:59.988046 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f48a6-e214-44c5-8563-0593967ef140 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:39:59.988264 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f48a6-e214-44c5-8563-0593967ef140 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:39:59.999451 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f48a6-e214-44c5-8563-0593967ef140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-215f48a6-e214-44c5-8563-0593967ef140 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 11:40:00.004204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:40:00.005255 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:00.006473 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 618/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Aug 30 11:39:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:40:00.011671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd695e-ef56-4713-8798-46a561ec4bf2 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:00.092300 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:00.113213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd695e-ef56-4713-8798-46a561ec4bf2 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:00.149906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e1a4-fa1d-46e9-9b47-d4ab8198ff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:00.150851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e1a4-fa1d-46e9-9b47-d4ab8198ff5b None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:00.151339 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e1a4-fa1d-46e9-9b47-d4ab8198ff5b None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:00.151777 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e1a4-fa1d-46e9-9b47-d4ab8198ff5b None admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:00.152853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917e1a4-fa1d-46e9-9b47-d4ab8198ff5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:00.153335 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917e1a4-fa1d-46e9-9b47-d4ab8198ff5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:00.155599 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917e1a4-fa1d-46e9-9b47-d4ab8198ff5b None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:00.168912 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 659/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Aug 30 11:40:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:40:00.174973 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f48a6-e214-44c5-8563-0593967ef140 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:00.279932 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f48a6-e214-44c5-8563-0593967ef140 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:00.314815 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19149b9-c675-4b65-b53c-8f855b404845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:00.315452 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19149b9-c675-4b65-b53c-8f855b404845 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:00.315721 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19149b9-c675-4b65-b53c-8f855b404845 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:00.315984 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19149b9-c675-4b65-b53c-8f855b404845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:00.326082 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19149b9-c675-4b65-b53c-8f855b404845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:00.327962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19149b9-c675-4b65-b53c-8f855b404845 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-b19149b9-c675-4b65-b53c-8f855b404845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:40:00.342292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:40:00.343501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:40:00.344587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:40:00.344587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:40:00.344587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:00.344587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 619/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:40:00.349801 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e1a4-fa1d-46e9-9b47-d4ab8198ff5b None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:00.459608 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e1a4-fa1d-46e9-9b47-d4ab8198ff5b None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:00.494092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1ed08-f0be-4c6e-a751-5f6cdb4a1892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:00.494686 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1ed08-f0be-4c6e-a751-5f6cdb4a1892 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:00.494960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1ed08-f0be-4c6e-a751-5f6cdb4a1892 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:00.495248 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1ed08-f0be-4c6e-a751-5f6cdb4a1892 None admin] PATH_INFO: `/v3/domains` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:00.495888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1ed08-f0be-4c6e-a751-5f6cdb4a1892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:00.496189 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1ed08-f0be-4c6e-a751-5f6cdb4a1892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:00.497547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1ed08-f0be-4c6e-a751-5f6cdb4a1892 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:00.510282 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 660/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:40:00.516113 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19149b9-c675-4b65-b53c-8f855b404845 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:00.549069 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:00.619837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19149b9-c675-4b65-b53c-8f855b404845 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:00.656703 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13280b93-54cc-45f4-9a0f-50956f25eede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:00.657310 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13280b93-54cc-45f4-9a0f-50956f25eede None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:00.657587 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13280b93-54cc-45f4-9a0f-50956f25eede None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:00.657852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13280b93-54cc-45f4-9a0f-50956f25eede None admin] PATH_INFO: `/v3/users/placement` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:00.665443 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13280b93-54cc-45f4-9a0f-50956f25eede None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:00.666878 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13280b93-54cc-45f4-9a0f-50956f25eede None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-13280b93-54cc-45f4-9a0f-50956f25eede None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 11:40:00.678065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:40:00.679172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:00.680197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 620/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:40:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:40:00.687242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1ed08-f0be-4c6e-a751-5f6cdb4a1892 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:00.691610 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:00.797965 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1ed08-f0be-4c6e-a751-5f6cdb4a1892 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:00.833191 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec7012-bb15-4c54-92ba-1ba8b716083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:00.833785 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec7012-bb15-4c54-92ba-1ba8b716083c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:00.834057 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec7012-bb15-4c54-92ba-1ba8b716083c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:00.834325 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec7012-bb15-4c54-92ba-1ba8b716083c None admin] PATH_INFO: `/v3/users` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:00.835047 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec7012-bb15-4c54-92ba-1ba8b716083c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:00.835380 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec7012-bb15-4c54-92ba-1ba8b716083c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:00.836803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec7012-bb15-4c54-92ba-1ba8b716083c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:00.853789 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 661/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Aug 30 11:40:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:40:00.860191 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13280b93-54cc-45f4-9a0f-50956f25eede None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:00.956585 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:00.971774 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13280b93-54cc-45f4-9a0f-50956f25eede None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:01.015731 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76ba1e-f6d7-4596-827a-142633fbb397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:01.016354 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76ba1e-f6d7-4596-827a-142633fbb397 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:01.016623 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76ba1e-f6d7-4596-827a-142633fbb397 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:01.016884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76ba1e-f6d7-4596-827a-142633fbb397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:01.089292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76ba1e-f6d7-4596-827a-142633fbb397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:01.090736 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76ba1e-f6d7-4596-827a-142633fbb397 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:01.179502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:01.190168 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 621/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Aug 30 11:40:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 11:40:01.199681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec7012-bb15-4c54-92ba-1ba8b716083c None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:01.270982 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:01.310889 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec7012-bb15-4c54-92ba-1ba8b716083c None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:01.346500 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4787ce-f8c1-413f-9702-542734f7f4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:01.347145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4787ce-f8c1-413f-9702-542734f7f4ef None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:01.347412 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4787ce-f8c1-413f-9702-542734f7f4ef None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:01.347675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4787ce-f8c1-413f-9702-542734f7f4ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:01.358560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4787ce-f8c1-413f-9702-542734f7f4ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:01.360023 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4787ce-f8c1-413f-9702-542734f7f4ef None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-5d4787ce-f8c1-413f-9702-542734f7f4ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 11:40:01.375923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:40:01.377069 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 11:40:01.378453 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 11:40:01.378453 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 11:40:01.378453 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:40:01.378453 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 662/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:40:01.384372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76ba1e-f6d7-4596-827a-142633fbb397 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:01.488292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76ba1e-f6d7-4596-827a-142633fbb397 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:01.523883 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b4b08-5f1c-4166-bdff-b9b6dd2f7e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:01.524478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b4b08-5f1c-4166-bdff-b9b6dd2f7e8a None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:01.524665 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b4b08-5f1c-4166-bdff-b9b6dd2f7e8a None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:01.524903 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b4b08-5f1c-4166-bdff-b9b6dd2f7e8a None admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:01.525558 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b4b08-5f1c-4166-bdff-b9b6dd2f7e8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:01.525856 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b4b08-5f1c-4166-bdff-b9b6dd2f7e8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:01.527239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b4b08-5f1c-4166-bdff-b9b6dd2f7e8a None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:01.538671 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 622/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Aug 30 11:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 11:40:01.547363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4787ce-f8c1-413f-9702-542734f7f4ef None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:01.649213 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4787ce-f8c1-413f-9702-542734f7f4ef None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:01.662759 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:01.686807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae66fd-6af4-4c4e-be35-67f067a405cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:01.687455 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae66fd-6af4-4c4e-be35-67f067a405cf None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:01.687725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae66fd-6af4-4c4e-be35-67f067a405cf None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:01.687989 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae66fd-6af4-4c4e-be35-67f067a405cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:01.694722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae66fd-6af4-4c4e-be35-67f067a405cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:01.696197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae66fd-6af4-4c4e-be35-67f067a405cf None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application [None req-cbae66fd-6af4-4c4e-be35-67f067a405cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 11:40:01.707754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application with Lock( Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:40:01.708861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:40:01.710203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 11:40:01.710203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 11:40:01.710203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 11:40:01.710203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 11:40:01.710203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 11:40:01.710203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 11:40:01.710203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 11:40:01.710203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: ERROR keystone.server.flask.application Aug 30 11:40:01.710203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 663/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Aug 30 11:40:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:40:01.717836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b4b08-5f1c-4166-bdff-b9b6dd2f7e8a None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:01.774432 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:01.829001 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b4b08-5f1c-4166-bdff-b9b6dd2f7e8a None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:01.863851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f7b0d-defd-46a9-b3b4-daa9ece42e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:01.864488 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f7b0d-defd-46a9-b3b4-daa9ece42e21 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:01.864757 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f7b0d-defd-46a9-b3b4-daa9ece42e21 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:01.865023 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f7b0d-defd-46a9-b3b4-daa9ece42e21 None admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:01.865709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f7b0d-defd-46a9-b3b4-daa9ece42e21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:01.866012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f7b0d-defd-46a9-b3b4-daa9ece42e21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:01.867467 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f7b0d-defd-46a9-b3b4-daa9ece42e21 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:01.879542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 623/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Aug 30 11:40:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:40:01.889247 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae66fd-6af4-4c4e-be35-67f067a405cf None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:02.006253 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae66fd-6af4-4c4e-be35-67f067a405cf None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:02.040096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d18db8-210a-49eb-908d-c1e5871208d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:02.040623 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d18db8-210a-49eb-908d-c1e5871208d7 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:02.040893 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d18db8-210a-49eb-908d-c1e5871208d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:02.041156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d18db8-210a-49eb-908d-c1e5871208d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:02.041860 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d18db8-210a-49eb-908d-c1e5871208d7 None admin] RBAC: Adding query filter params (role.id=0209018e60254b91b5b7ea8d0bc5d501, scope.project.id=c0027f77ae674bc08f99838ef869845c, user.id=c8208192d40243e9a81afb254a9326b8) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:02.042151 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d18db8-210a-49eb-908d-c1e5871208d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:02.043650 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d18db8-210a-49eb-908d-c1e5871208d7 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:02.049320 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 664/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Aug 30 11:40:01 2022] GET /identity/v3/role_assignments?user.id=c8208192d40243e9a81afb254a9326b8&scope.project.id=c0027f77ae674bc08f99838ef869845c&role.id=0209018e60254b91b5b7ea8d0bc5d501 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:40:03.935936 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a579b9-c79b-41bf-811a-61457365661a None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:03.936521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a579b9-c79b-41bf-811a-61457365661a None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:03.936521 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a579b9-c79b-41bf-811a-61457365661a None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:03.937857 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 624/1288] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:40:03.944342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c80d-27ad-470b-af90-af132998fbe6 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:03.945075 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c80d-27ad-470b-af90-af132998fbe6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:03.945232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c80d-27ad-470b-af90-af132998fbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:03.970086 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-ca68c80d-27ad-470b-af90-af132998fbe6 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:40:03.971510 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ca68c80d-27ad-470b-af90-af132998fbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:40:04.006396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:04.051058 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 665/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:40:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:40:04.316107 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a579b9-c79b-41bf-811a-61457365661a None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:04.420545 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a579b9-c79b-41bf-811a-61457365661a None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:04.431626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:04.457273 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119645d-0301-4e55-be09-20e48fcc1828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:04.457903 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119645d-0301-4e55-be09-20e48fcc1828 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:04.458177 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119645d-0301-4e55-be09-20e48fcc1828 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:04.458436 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119645d-0301-4e55-be09-20e48fcc1828 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:04.459003 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119645d-0301-4e55-be09-20e48fcc1828 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:04.460408 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119645d-0301-4e55-be09-20e48fcc1828 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-9119645d-0301-4e55-be09-20e48fcc1828 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:40:04.469358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:40:04.470570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:04.471583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 625/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:40:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:40:04.476798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c80d-27ad-470b-af90-af132998fbe6 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:04.574357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c80d-27ad-470b-af90-af132998fbe6 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:04.610161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067abaf5-91a4-40a1-8ded-0ff6e873043c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:04.610750 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067abaf5-91a4-40a1-8ded-0ff6e873043c None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:04.611021 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067abaf5-91a4-40a1-8ded-0ff6e873043c None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:04.611314 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067abaf5-91a4-40a1-8ded-0ff6e873043c None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:04.611937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067abaf5-91a4-40a1-8ded-0ff6e873043c None admin] RBAC: Adding query filter params (name=placement) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:04.612237 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067abaf5-91a4-40a1-8ded-0ff6e873043c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:04.613595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067abaf5-91a4-40a1-8ded-0ff6e873043c None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:04.618466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 666/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Aug 30 11:40:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:40:04.624456 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119645d-0301-4e55-be09-20e48fcc1828 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:04.727823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119645d-0301-4e55-be09-20e48fcc1828 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:04.763838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d9cdf-95de-42f7-a1ab-00bf3b0cb247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:04.764423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d9cdf-95de-42f7-a1ab-00bf3b0cb247 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:04.764743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d9cdf-95de-42f7-a1ab-00bf3b0cb247 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:04.765017 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d9cdf-95de-42f7-a1ab-00bf3b0cb247 None admin] PATH_INFO: `/v3/services` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:04.765643 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d9cdf-95de-42f7-a1ab-00bf3b0cb247 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:04.765936 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d9cdf-95de-42f7-a1ab-00bf3b0cb247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:04.767330 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d9cdf-95de-42f7-a1ab-00bf3b0cb247 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:04.772031 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 626/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Aug 30 11:40:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:40:06.626787 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a59db-4943-498b-8c06-3359e28b6962 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:06.627673 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a59db-4943-498b-8c06-3359e28b6962 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:06.627673 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a59db-4943-498b-8c06-3359e28b6962 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:06.629846 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 667/1293] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:40:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:40:06.637611 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b6dbb-de4b-4458-ae58-008279f9b1ab None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:06.638071 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b6dbb-de4b-4458-ae58-008279f9b1ab None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:06.638172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b6dbb-de4b-4458-ae58-008279f9b1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:06.658021 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-d60b6dbb-de4b-4458-ae58-008279f9b1ab None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:40:06.659213 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-d60b6dbb-de4b-4458-ae58-008279f9b1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:40:06.689212 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:06.727780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 627/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:40:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 11:40:07.008990 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a59db-4943-498b-8c06-3359e28b6962 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:07.049415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:07.129227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a59db-4943-498b-8c06-3359e28b6962 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:07.163929 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdcd28-835b-48c3-8e90-f0608d8fc599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:07.164537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdcd28-835b-48c3-8e90-f0608d8fc599 None admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:07.164813 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdcd28-835b-48c3-8e90-f0608d8fc599 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:07.165081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdcd28-835b-48c3-8e90-f0608d8fc599 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:07.165742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdcd28-835b-48c3-8e90-f0608d8fc599 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:07.167149 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdcd28-835b-48c3-8e90-f0608d8fc599 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:07.183438 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 668/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Aug 30 11:40:07 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 11:40:09.097983 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbbf6e-a0c3-46b8-8489-ae0bc381a8d7 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:09.098570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbbf6e-a0c3-46b8-8489-ae0bc381a8d7 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:09.098570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbbf6e-a0c3-46b8-8489-ae0bc381a8d7 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:09.100047 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 628/1296] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:40:09.106224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392748b-fa08-49f2-955c-49b721c40ea1 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:09.106979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392748b-fa08-49f2-955c-49b721c40ea1 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:09.107176 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392748b-fa08-49f2-955c-49b721c40ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:09.128420 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-0392748b-fa08-49f2-955c-49b721c40ea1 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:40:09.129594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-0392748b-fa08-49f2-955c-49b721c40ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:40:09.208801 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 669/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:40:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:40:09.468841 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbbf6e-a0c3-46b8-8489-ae0bc381a8d7 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:09.588994 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbbf6e-a0c3-46b8-8489-ae0bc381a8d7 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:09.624244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89700822-adef-48ec-a6ef-979804e9f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:09.624851 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89700822-adef-48ec-a6ef-979804e9f601 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:09.625119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89700822-adef-48ec-a6ef-979804e9f601 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:09.625366 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89700822-adef-48ec-a6ef-979804e9f601 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:09.625938 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89700822-adef-48ec-a6ef-979804e9f601 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:09.627336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89700822-adef-48ec-a6ef-979804e9f601 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-89700822-adef-48ec-a6ef-979804e9f601 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:40:09.635875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:40:09.637040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:09.638249 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 629/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:40:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 11:40:09.644978 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392748b-fa08-49f2-955c-49b721c40ea1 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:09.668111 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:09.749712 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392748b-fa08-49f2-955c-49b721c40ea1 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:09.784153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4ab14-7c40-4a4b-a8c2-cee557883ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:09.784792 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4ab14-7c40-4a4b-a8c2-cee557883ec9 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:09.785064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4ab14-7c40-4a4b-a8c2-cee557883ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:09.785322 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4ab14-7c40-4a4b-a8c2-cee557883ec9 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:09.785989 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4ab14-7c40-4a4b-a8c2-cee557883ec9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:09.786291 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4ab14-7c40-4a4b-a8c2-cee557883ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:09.787730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4ab14-7c40-4a4b-a8c2-cee557883ec9 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:09.792874 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 670/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Aug 30 11:40:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:40:09.798899 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89700822-adef-48ec-a6ef-979804e9f601 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:09.833232 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:09.904144 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89700822-adef-48ec-a6ef-979804e9f601 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:09.939342 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb20935-107d-4b99-9ff4-02bed490b960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:09.939943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb20935-107d-4b99-9ff4-02bed490b960 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:09.940218 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb20935-107d-4b99-9ff4-02bed490b960 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:09.940486 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb20935-107d-4b99-9ff4-02bed490b960 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:09.941202 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb20935-107d-4b99-9ff4-02bed490b960 None admin] RBAC: Adding query filter params (interface=public, service_id=8b54710687214244a1f831e77f1f899c, region_id=RegionOne) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:09.941509 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb20935-107d-4b99-9ff4-02bed490b960 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:09.942886 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb20935-107d-4b99-9ff4-02bed490b960 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:09.948104 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 630/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Aug 30 11:40:09 2022] GET /identity/v3/endpoints?service_id=8b54710687214244a1f831e77f1f899c&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:40:09.954384 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4ab14-7c40-4a4b-a8c2-cee557883ec9 None admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:10.053528 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:10.074634 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4ab14-7c40-4a4b-a8c2-cee557883ec9 None admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:10.110599 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8a4a0-4fa2-4886-9363-b04027126c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:10.111217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8a4a0-4fa2-4886-9363-b04027126c61 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:10.111523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8a4a0-4fa2-4886-9363-b04027126c61 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:10.111787 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8a4a0-4fa2-4886-9363-b04027126c61 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:10.112385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8a4a0-4fa2-4886-9363-b04027126c61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:10.113754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8a4a0-4fa2-4886-9363-b04027126c61 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:10.120570 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 671/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Aug 30 11:40:09 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 11:40:11.988198 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707bc43-9595-4a81-9bd5-e024b249bfa0 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:11.988757 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707bc43-9595-4a81-9bd5-e024b249bfa0 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:11.988757 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707bc43-9595-4a81-9bd5-e024b249bfa0 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:11.990078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 631/1302] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:40:11.995518 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68646355-64c6-4988-9e8d-40826de7f830 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:11.995923 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68646355-64c6-4988-9e8d-40826de7f830 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:11.996052 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68646355-64c6-4988-9e8d-40826de7f830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:12.015485 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-68646355-64c6-4988-9e8d-40826de7f830 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:40:12.016678 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-68646355-64c6-4988-9e8d-40826de7f830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:40:12.090239 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 672/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:40:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 11:40:12.358384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707bc43-9595-4a81-9bd5-e024b249bfa0 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:12.463461 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:12.471743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707bc43-9595-4a81-9bd5-e024b249bfa0 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:12.507149 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef42eac-94f8-4d8f-afa8-d52f859d7317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:12.507818 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef42eac-94f8-4d8f-afa8-d52f859d7317 None admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:12.508120 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef42eac-94f8-4d8f-afa8-d52f859d7317 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:12.508388 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef42eac-94f8-4d8f-afa8-d52f859d7317 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:12.508967 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef42eac-94f8-4d8f-afa8-d52f859d7317 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:12.510358 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef42eac-94f8-4d8f-afa8-d52f859d7317 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application [None req-9ef42eac-94f8-4d8f-afa8-d52f859d7317 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 11:40:12.519946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 11:40:12.521130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application with Lock( Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self._enter() Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application return self.creator() Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application created_value = creator( Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: ERROR keystone.server.flask.application Aug 30 11:40:12.522327 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 632/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Aug 30 11:40:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 11:40:12.527849 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68646355-64c6-4988-9e8d-40826de7f830 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:12.612441 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:12.641221 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68646355-64c6-4988-9e8d-40826de7f830 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:12.684346 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9719a-4047-4324-b0f7-ceda25023b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:12.685242 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9719a-4047-4324-b0f7-ceda25023b19 None admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:12.685642 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9719a-4047-4324-b0f7-ceda25023b19 None admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:12.686064 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9719a-4047-4324-b0f7-ceda25023b19 None admin] PATH_INFO: `/v3/services` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:12.686794 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9719a-4047-4324-b0f7-ceda25023b19 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:40:12.687090 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9719a-4047-4324-b0f7-ceda25023b19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:12.688560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9719a-4047-4324-b0f7-ceda25023b19 None admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:12.694091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 673/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Aug 30 11:40:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:40:12.700092 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef42eac-94f8-4d8f-afa8-d52f859d7317 None admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:40:12.813078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef42eac-94f8-4d8f-afa8-d52f859d7317 None admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:40:12.855016 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65d053-1362-4a42-8458-038683804755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:40:12.855943 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65d053-1362-4a42-8458-038683804755 None admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:40:12.856312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65d053-1362-4a42-8458-038683804755 None admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:40:12.856669 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65d053-1362-4a42-8458-038683804755 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:40:12.857673 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65d053-1362-4a42-8458-038683804755 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:40:12.859784 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65d053-1362-4a42-8458-038683804755 None admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:40:12.883029 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 633/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:40:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:40:22.690825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=30 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:40:22.878512 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:16.345030 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adbea3-fbbd-4924-83e8-0a0ff810b3d1 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:16.345549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adbea3-fbbd-4924-83e8-0a0ff810b3d1 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:16.345549 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adbea3-fbbd-4924-83e8-0a0ff810b3d1 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:16.347091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 674/1307] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:16.352633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b1c9c-a350-4051-ac7c-bba05e62d3f4 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:16.353047 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b1c9c-a350-4051-ac7c-bba05e62d3f4 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:16.353208 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b1c9c-a350-4051-ac7c-bba05e62d3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:16.510115 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-2c4b1c9c-a350-4051-ac7c-bba05e62d3f4 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:16.511809 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-2c4b1c9c-a350-4051-ac7c-bba05e62d3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:16.676514 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 634/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 11:41:16.968277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e3a90-908d-4f7d-b0e9-b8ac6e538a08 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:16.968634 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e3a90-908d-4f7d-b0e9-b8ac6e538a08 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:16.968840 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e3a90-908d-4f7d-b0e9-b8ac6e538a08 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:16.970610 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 675/1309] 10.222.0.16 () {54 vars in 965 bytes} [Tue Aug 30 11:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:16.978051 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbb9f5-deb6-4230-91fa-0b1d5c873640 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:16.978321 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbb9f5-deb6-4230-91fa-0b1d5c873640 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:16.978873 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbb9f5-deb6-4230-91fa-0b1d5c873640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:17.064934 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-7edbb9f5-deb6-4230-91fa-0b1d5c873640 None None] MFA Rules not processed for user `56bc4fdd14d147ef86b96786da2855f5`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:17.066175 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-7edbb9f5-deb6-4230-91fa-0b1d5c873640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:17.089780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:17.155157 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 635/1310] 10.222.0.16 () {58 vars in 1062 bytes} [Tue Aug 30 11:41:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:17.163932 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e3a90-908d-4f7d-b0e9-b8ac6e538a08 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:17.165646 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-892e3a90-908d-4f7d-b0e9-b8ac6e538a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:17.275587 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e3a90-908d-4f7d-b0e9-b8ac6e538a08 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:17.351133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e9981-e0b0-46f4-91de-8be0862462f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:17.351762 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e9981-e0b0-46f4-91de-8be0862462f2 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:17.351993 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e9981-e0b0-46f4-91de-8be0862462f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:17.352123 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e9981-e0b0-46f4-91de-8be0862462f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:17.353716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-956e9981-e0b0-46f4-91de-8be0862462f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:17.418750 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:17.471837 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e9981-e0b0-46f4-91de-8be0862462f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:17.473293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e9981-e0b0-46f4-91de-8be0862462f2 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:17.492948 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 676/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:19.691015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885846b-3a34-4394-afb5-135254a066dc None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:19.691540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885846b-3a34-4394-afb5-135254a066dc None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:19.691540 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885846b-3a34-4394-afb5-135254a066dc None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:19.693361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 636/1312] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:19.698424 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616d64-2d76-4c88-99a3-6cb829219b69 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:19.698875 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616d64-2d76-4c88-99a3-6cb829219b69 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:19.698959 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616d64-2d76-4c88-99a3-6cb829219b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:19.773788 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-3d616d64-2d76-4c88-99a3-6cb829219b69 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:19.774988 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-3d616d64-2d76-4c88-99a3-6cb829219b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:19.879830 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 677/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:20.162052 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885846b-3a34-4394-afb5-135254a066dc None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:20.164543 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-b885846b-3a34-4394-afb5-135254a066dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:20.282707 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885846b-3a34-4394-afb5-135254a066dc None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:20.355949 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:20.359082 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4114f85-364d-46db-b807-776ec82d04c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:20.359964 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114f85-364d-46db-b807-776ec82d04c9 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:20.360469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114f85-364d-46db-b807-776ec82d04c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:20.360737 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114f85-364d-46db-b807-776ec82d04c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:20.362591 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c4114f85-364d-46db-b807-776ec82d04c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:20.480374 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4114f85-364d-46db-b807-776ec82d04c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:20.481927 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4114f85-364d-46db-b807-776ec82d04c9 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:20.500969 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 637/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:22.687633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993aff4-2c52-4997-a322-c5a51cd93653 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:22.688210 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993aff4-2c52-4997-a322-c5a51cd93653 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:22.688210 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993aff4-2c52-4997-a322-c5a51cd93653 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:22.689461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 678/1315] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:22.695759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62131bd-bfd7-4221-996c-48b9ddc17733 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:22.696186 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62131bd-bfd7-4221-996c-48b9ddc17733 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:22.696314 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62131bd-bfd7-4221-996c-48b9ddc17733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:22.771012 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-d62131bd-bfd7-4221-996c-48b9ddc17733 None None] MFA Rules not processed for user `3092900ab6da4b199709df63140b22ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:22.772253 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-d62131bd-bfd7-4221-996c-48b9ddc17733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:22.824195 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:22.874035 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 638/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:23.155070 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993aff4-2c52-4997-a322-c5a51cd93653 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:23.156632 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f993aff4-2c52-4997-a322-c5a51cd93653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:23.167709 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:23.266905 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993aff4-2c52-4997-a322-c5a51cd93653 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:23.341309 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e076d4-5e05-431d-8ea4-910c68cb3125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:23.341906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e076d4-5e05-431d-8ea4-910c68cb3125 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:23.342178 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e076d4-5e05-431d-8ea4-910c68cb3125 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:23.342435 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e076d4-5e05-431d-8ea4-910c68cb3125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:23.344029 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-33e076d4-5e05-431d-8ea4-910c68cb3125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:23.410704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:23.458603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e076d4-5e05-431d-8ea4-910c68cb3125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:23.460089 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e076d4-5e05-431d-8ea4-910c68cb3125 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:23.479334 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 679/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:25.919955 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e5521-4f9a-40b1-934e-f588f13605e0 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:25.920410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e5521-4f9a-40b1-934e-f588f13605e0 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:25.920504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e5521-4f9a-40b1-934e-f588f13605e0 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:25.922613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 639/1318] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:25.931031 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00175fab-135d-4603-a86f-7395b133e6c3 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:25.931450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00175fab-135d-4603-a86f-7395b133e6c3 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:25.931595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00175fab-135d-4603-a86f-7395b133e6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:26.006579 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-00175fab-135d-4603-a86f-7395b133e6c3 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:26.007835 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-00175fab-135d-4603-a86f-7395b133e6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:26.111193 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 680/1319] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:26.394202 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e5521-4f9a-40b1-934e-f588f13605e0 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:26.396649 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-fc1e5521-4f9a-40b1-934e-f588f13605e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:26.508791 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e5521-4f9a-40b1-934e-f588f13605e0 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:26.591730 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a70fa-24cd-43e6-a589-a8bfe2f7b3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:26.592323 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a70fa-24cd-43e6-a589-a8bfe2f7b3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:26.592588 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a70fa-24cd-43e6-a589-a8bfe2f7b3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:26.592848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a70fa-24cd-43e6-a589-a8bfe2f7b3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:26.594420 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-2b9a70fa-24cd-43e6-a589-a8bfe2f7b3e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:26.601328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:26.709167 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a70fa-24cd-43e6-a589-a8bfe2f7b3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:26.710511 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a70fa-24cd-43e6-a589-a8bfe2f7b3e3 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:26.728237 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 640/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:29.263139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf2c66-fa5b-494d-a99a-66a00f3029d8 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:29.263694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf2c66-fa5b-494d-a99a-66a00f3029d8 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:29.263819 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf2c66-fa5b-494d-a99a-66a00f3029d8 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:29.265162 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 681/1321] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:29.270757 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ca8f5-a4d5-4a25-939c-8f590366b866 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:29.271197 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ca8f5-a4d5-4a25-939c-8f590366b866 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:29.271312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ca8f5-a4d5-4a25-939c-8f590366b866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:29.341501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-894ca8f5-a4d5-4a25-939c-8f590366b866 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:29.342707 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-894ca8f5-a4d5-4a25-939c-8f590366b866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:29.404097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:29.447391 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 641/1322] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:29.727609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf2c66-fa5b-494d-a99a-66a00f3029d8 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:29.729149 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ffbf2c66-fa5b-494d-a99a-66a00f3029d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:29.749238 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:29.836752 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf2c66-fa5b-494d-a99a-66a00f3029d8 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:29.911382 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01867ba6-d1c8-46a0-94c7-2dac2161b74d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:29.911984 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01867ba6-d1c8-46a0-94c7-2dac2161b74d service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:29.912262 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01867ba6-d1c8-46a0-94c7-2dac2161b74d service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:29.912527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01867ba6-d1c8-46a0-94c7-2dac2161b74d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:29.914104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-01867ba6-d1c8-46a0-94c7-2dac2161b74d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:29.992401 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:30.031307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01867ba6-d1c8-46a0-94c7-2dac2161b74d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:30.032770 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01867ba6-d1c8-46a0-94c7-2dac2161b74d service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:30.051306 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 682/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:32.019517 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbb4a2-430b-410d-aba1-9be219d6e552 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:32.020066 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbb4a2-430b-410d-aba1-9be219d6e552 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:32.020066 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbb4a2-430b-410d-aba1-9be219d6e552 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:32.021417 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 642/1324] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:32.026851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff95f4-9552-45e9-b152-41a889f02964 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:32.027390 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff95f4-9552-45e9-b152-41a889f02964 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:32.027531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff95f4-9552-45e9-b152-41a889f02964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:32.096702 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-84ff95f4-9552-45e9-b152-41a889f02964 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:32.097888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-84ff95f4-9552-45e9-b152-41a889f02964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:32.201366 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 683/1325] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:32.490625 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91072ca0-eb34-4ce2-89d5-1f6f9bb1d090 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:32.490906 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91072ca0-eb34-4ce2-89d5-1f6f9bb1d090 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:32.491241 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91072ca0-eb34-4ce2-89d5-1f6f9bb1d090 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:32.492590 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 643/1326] 10.222.0.16 () {54 vars in 965 bytes} [Tue Aug 30 11:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:32.499038 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f91fb-58e2-49d8-951f-b2a58287e082 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:32.499367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f91fb-58e2-49d8-951f-b2a58287e082 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:32.499635 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f91fb-58e2-49d8-951f-b2a58287e082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:32.541418 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:32.587085 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-f72f91fb-58e2-49d8-951f-b2a58287e082 None None] MFA Rules not processed for user `56bc4fdd14d147ef86b96786da2855f5`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:32.588308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f72f91fb-58e2-49d8-951f-b2a58287e082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:32.664627 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 684/1327] 10.222.0.16 () {58 vars in 1062 bytes} [Tue Aug 30 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:32.673507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91072ca0-eb34-4ce2-89d5-1f6f9bb1d090 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:32.675002 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-91072ca0-eb34-4ce2-89d5-1f6f9bb1d090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:32.779625 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91072ca0-eb34-4ce2-89d5-1f6f9bb1d090 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:32.855718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fad1bc-f07d-4fdc-8d94-3eaeee3425ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:32.856231 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fad1bc-f07d-4fdc-8d94-3eaeee3425ba service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:32.856500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fad1bc-f07d-4fdc-8d94-3eaeee3425ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:32.856765 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fad1bc-f07d-4fdc-8d94-3eaeee3425ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:32.858320 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c1fad1bc-f07d-4fdc-8d94-3eaeee3425ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:32.875000 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:32.968925 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fad1bc-f07d-4fdc-8d94-3eaeee3425ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:32.970363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fad1bc-f07d-4fdc-8d94-3eaeee3425ba service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:32.991100 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 644/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:35.147574 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162b28-71e9-422c-a227-48bf0ca05cd1 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:35.148185 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162b28-71e9-422c-a227-48bf0ca05cd1 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:35.148185 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162b28-71e9-422c-a227-48bf0ca05cd1 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:35.149462 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 685/1329] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:35.154906 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea4d19-5f65-45ee-81ac-7a6d471100ec None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:35.155410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea4d19-5f65-45ee-81ac-7a6d471100ec None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:35.155513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea4d19-5f65-45ee-81ac-7a6d471100ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:35.227806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-97ea4d19-5f65-45ee-81ac-7a6d471100ec None None] MFA Rules not processed for user `3092900ab6da4b199709df63140b22ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:35.229009 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-97ea4d19-5f65-45ee-81ac-7a6d471100ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:35.300444 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:35.334811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 645/1330] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:35.617907 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a162b28-71e9-422c-a227-48bf0ca05cd1 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:35.619449 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-3a162b28-71e9-422c-a227-48bf0ca05cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:35.737363 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a162b28-71e9-422c-a227-48bf0ca05cd1 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:35.758537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:35.818317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f810c-635f-4467-8c6e-8382d1c72f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:35.818937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f810c-635f-4467-8c6e-8382d1c72f44 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:35.819240 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f810c-635f-4467-8c6e-8382d1c72f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:35.819502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f810c-635f-4467-8c6e-8382d1c72f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:35.821145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-281f810c-635f-4467-8c6e-8382d1c72f44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:35.942187 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f810c-635f-4467-8c6e-8382d1c72f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:35.945148 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f810c-635f-4467-8c6e-8382d1c72f44 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:35.966453 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 686/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 11:41:38.238015 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6514825-d3ce-4042-bdf3-2674a7228487 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:38.238581 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6514825-d3ce-4042-bdf3-2674a7228487 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:38.238581 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6514825-d3ce-4042-bdf3-2674a7228487 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:38.239958 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 646/1332] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:38.246079 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cef198-79b4-4a0e-97bd-fa5ee1ef59a6 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:38.246511 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cef198-79b4-4a0e-97bd-fa5ee1ef59a6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:38.246601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cef198-79b4-4a0e-97bd-fa5ee1ef59a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:38.325239 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:38.329002 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-46cef198-79b4-4a0e-97bd-fa5ee1ef59a6 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:38.331062 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-46cef198-79b4-4a0e-97bd-fa5ee1ef59a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:38.432753 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 687/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:38.709304 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6514825-d3ce-4042-bdf3-2674a7228487 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:38.711797 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-e6514825-d3ce-4042-bdf3-2674a7228487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:38.824192 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6514825-d3ce-4042-bdf3-2674a7228487 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:38.904687 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622ebf9-614f-4c98-9088-dbd535e71c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:38.905297 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622ebf9-614f-4c98-9088-dbd535e71c39 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:38.905565 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622ebf9-614f-4c98-9088-dbd535e71c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:38.905831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622ebf9-614f-4c98-9088-dbd535e71c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:38.907479 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-f622ebf9-614f-4c98-9088-dbd535e71c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:38.934858 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:39.044603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622ebf9-614f-4c98-9088-dbd535e71c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:39.046067 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622ebf9-614f-4c98-9088-dbd535e71c39 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:39.063760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 647/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:40.994152 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65e7c0-1aea-48af-a2ad-998a63bc847c None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:40.994734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65e7c0-1aea-48af-a2ad-998a63bc847c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:40.994734 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65e7c0-1aea-48af-a2ad-998a63bc847c None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:40.996110 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 688/1335] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:41.001337 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c36839-0a93-43cb-afdd-c48f2ddb2065 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:41.001812 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c36839-0a93-43cb-afdd-c48f2ddb2065 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:41.001925 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c36839-0a93-43cb-afdd-c48f2ddb2065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:41.075238 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-e8c36839-0a93-43cb-afdd-c48f2ddb2065 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:41.077367 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-e8c36839-0a93-43cb-afdd-c48f2ddb2065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:41.157353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:41.179795 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 648/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:41.460844 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65e7c0-1aea-48af-a2ad-998a63bc847c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:41.463265 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-1b65e7c0-1aea-48af-a2ad-998a63bc847c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:41.580194 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65e7c0-1aea-48af-a2ad-998a63bc847c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:41.608812 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:41.654958 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00db76-bbd0-4a52-aff3-17c1d7d3bcb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:41.655597 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00db76-bbd0-4a52-aff3-17c1d7d3bcb8 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:41.655867 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00db76-bbd0-4a52-aff3-17c1d7d3bcb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:41.656131 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00db76-bbd0-4a52-aff3-17c1d7d3bcb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:41.657715 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-2b00db76-bbd0-4a52-aff3-17c1d7d3bcb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:41.767972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00db76-bbd0-4a52-aff3-17c1d7d3bcb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:41.769403 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00db76-bbd0-4a52-aff3-17c1d7d3bcb8 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:41.790878 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 689/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:44.137331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8fcd7-01d9-4873-ab66-1231722177e7 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:44.137909 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8fcd7-01d9-4873-ab66-1231722177e7 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:44.137909 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8fcd7-01d9-4873-ab66-1231722177e7 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:44.139207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 649/1338] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:44.145588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4f205-8b66-4bca-8912-c7dd0c47d77a None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:44.146361 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4f205-8b66-4bca-8912-c7dd0c47d77a None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:44.146511 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4f205-8b66-4bca-8912-c7dd0c47d77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:44.218901 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-41f4f205-8b66-4bca-8912-c7dd0c47d77a None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:44.220040 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-41f4f205-8b66-4bca-8912-c7dd0c47d77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:44.232316 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:44.339530 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 690/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:44.615512 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8fcd7-01d9-4873-ab66-1231722177e7 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:44.617030 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-18b8fcd7-01d9-4873-ab66-1231722177e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:44.743854 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8fcd7-01d9-4873-ab66-1231722177e7 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:44.817239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bee86e-6ee6-4bc8-8045-6c7098818103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:44.817833 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bee86e-6ee6-4bc8-8045-6c7098818103 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:44.818099 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bee86e-6ee6-4bc8-8045-6c7098818103 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:44.818369 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bee86e-6ee6-4bc8-8045-6c7098818103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:44.819962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-a3bee86e-6ee6-4bc8-8045-6c7098818103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:44.857870 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:44.936556 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bee86e-6ee6-4bc8-8045-6c7098818103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:44.937997 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bee86e-6ee6-4bc8-8045-6c7098818103 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:44.956208 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 650/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:48.917293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58353dae-97d9-4008-92d4-f8a2ae401c1c None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:48.917766 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58353dae-97d9-4008-92d4-f8a2ae401c1c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:48.917862 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58353dae-97d9-4008-92d4-f8a2ae401c1c None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:48.919208 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 691/1341] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:48.924583 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f841f2-f92d-45e5-a1fc-228fbc20f4c2 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:48.925042 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f841f2-f92d-45e5-a1fc-228fbc20f4c2 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:48.925189 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f841f2-f92d-45e5-a1fc-228fbc20f4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:48.993671 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-73f841f2-f92d-45e5-a1fc-228fbc20f4c2 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:48.994874 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-73f841f2-f92d-45e5-a1fc-228fbc20f4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:49.082762 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:49.095478 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 651/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:49.373328 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58353dae-97d9-4008-92d4-f8a2ae401c1c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:49.374844 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-58353dae-97d9-4008-92d4-f8a2ae401c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:49.484684 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58353dae-97d9-4008-92d4-f8a2ae401c1c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:49.523138 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:49.559935 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a927e4-71a6-4df4-9b6a-72c02404f26c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:49.560547 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a927e4-71a6-4df4-9b6a-72c02404f26c service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:49.560815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a927e4-71a6-4df4-9b6a-72c02404f26c service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:49.561074 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a927e4-71a6-4df4-9b6a-72c02404f26c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:49.562660 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-34a927e4-71a6-4df4-9b6a-72c02404f26c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:49.675432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a927e4-71a6-4df4-9b6a-72c02404f26c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:49.676784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a927e4-71a6-4df4-9b6a-72c02404f26c service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:49.695166 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 692/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:52.195034 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daea701-233c-4c59-ac9b-74ea06b4b1fd None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:52.195600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daea701-233c-4c59-ac9b-74ea06b4b1fd None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:52.195600 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daea701-233c-4c59-ac9b-74ea06b4b1fd None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:52.196895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 652/1344] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:52.202920 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93dd09-99cc-4da7-a510-f0aa7cf29871 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:52.203471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93dd09-99cc-4da7-a510-f0aa7cf29871 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:52.203471 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93dd09-99cc-4da7-a510-f0aa7cf29871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:52.272870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-fe93dd09-99cc-4da7-a510-f0aa7cf29871 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:52.274058 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-fe93dd09-99cc-4da7-a510-f0aa7cf29871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:52.292575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:52.375583 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 693/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:52.649201 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daea701-233c-4c59-ac9b-74ea06b4b1fd None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:52.650792 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-9daea701-233c-4c59-ac9b-74ea06b4b1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:52.754743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daea701-233c-4c59-ac9b-74ea06b4b1fd None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:52.828504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0eda3-a4df-48ee-8e20-6c95bcf0c964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:52.829102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0eda3-a4df-48ee-8e20-6c95bcf0c964 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:52.829368 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0eda3-a4df-48ee-8e20-6c95bcf0c964 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:52.829627 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0eda3-a4df-48ee-8e20-6c95bcf0c964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:52.831223 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-2ad0eda3-a4df-48ee-8e20-6c95bcf0c964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:52.874714 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:52.949655 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0eda3-a4df-48ee-8e20-6c95bcf0c964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:52.951338 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0eda3-a4df-48ee-8e20-6c95bcf0c964 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:52.970131 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 653/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:41:56.799777 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be58bd-ac45-4266-8147-92db9632e120 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:56.800260 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be58bd-ac45-4266-8147-92db9632e120 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:56.800260 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be58bd-ac45-4266-8147-92db9632e120 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:56.801661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 694/1347] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:41:56.807253 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8358b-a05f-4274-b4fc-46a81a1762e8 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:56.807891 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8358b-a05f-4274-b4fc-46a81a1762e8 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:56.807891 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8358b-a05f-4274-b4fc-46a81a1762e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:56.890487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-d9b8358b-a05f-4274-b4fc-46a81a1762e8 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:41:56.891708 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-d9b8358b-a05f-4274-b4fc-46a81a1762e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:56.998097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:57.001003 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 654/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:41:57.275353 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be58bd-ac45-4266-8147-92db9632e120 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:41:57.276925 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-63be58bd-ac45-4266-8147-92db9632e120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:57.386154 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be58bd-ac45-4266-8147-92db9632e120 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:41:57.438817 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:41:57.465575 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200523db-79ce-4591-b828-a93b70cb94b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:41:57.466114 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200523db-79ce-4591-b828-a93b70cb94b3 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:41:57.466377 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200523db-79ce-4591-b828-a93b70cb94b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:41:57.466646 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200523db-79ce-4591-b828-a93b70cb94b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:41:57.468349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-200523db-79ce-4591-b828-a93b70cb94b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:41:57.579442 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200523db-79ce-4591-b828-a93b70cb94b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:41:57.580903 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200523db-79ce-4591-b828-a93b70cb94b3 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:41:57.599165 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 695/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:02.422142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2ad11-85bb-4962-952b-870d9f0e9d1c None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:02.422656 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2ad11-85bb-4962-952b-870d9f0e9d1c None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:02.422656 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2ad11-85bb-4962-952b-870d9f0e9d1c None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:02.424634 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 655/1350] 10.222.0.16 () {54 vars in 921 bytes} [Tue Aug 30 11:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:02.429772 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d015ef-9459-48fd-9635-3bf1e827f2e0 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:02.430205 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d015ef-9459-48fd-9635-3bf1e827f2e0 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:02.430700 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d015ef-9459-48fd-9635-3bf1e827f2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:02.524480 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-42d015ef-9459-48fd-9635-3bf1e827f2e0 None None] MFA Rules not processed for user `c8208192d40243e9a81afb254a9326b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:02.526110 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-42d015ef-9459-48fd-9635-3bf1e827f2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:02.553025 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:02.681648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 696/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Aug 30 11:42:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:06.774532 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ee4b8-ab5f-413f-813b-bcf94c495d4f None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:06.775315 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ee4b8-ab5f-413f-813b-bcf94c495d4f None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:06.775315 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ee4b8-ab5f-413f-813b-bcf94c495d4f None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:06.776690 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 656/1352] 10.222.0.16 () {54 vars in 921 bytes} [Tue Aug 30 11:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:42:06.781451 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4d241-2a00-431a-817a-23885dac716a None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:06.781928 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4d241-2a00-431a-817a-23885dac716a None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:06.782183 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4d241-2a00-431a-817a-23885dac716a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:06.865501 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-e3a4d241-2a00-431a-817a-23885dac716a None None] MFA Rules not processed for user `c8208192d40243e9a81afb254a9326b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:06.866859 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-e3a4d241-2a00-431a-817a-23885dac716a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:06.940581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:06.978155 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 697/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Aug 30 11:42:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:08.324698 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc9bfc-de47-486e-8f50-305f142602b2 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:08.325588 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc9bfc-de47-486e-8f50-305f142602b2 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:08.325588 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc9bfc-de47-486e-8f50-305f142602b2 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:08.326918 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 657/1354] 10.222.0.16 () {54 vars in 921 bytes} [Tue Aug 30 11:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:08.332317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251b8c8-c3cf-48b4-881b-fb36b95e1125 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:08.332749 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251b8c8-c3cf-48b4-881b-fb36b95e1125 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:08.332853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251b8c8-c3cf-48b4-881b-fb36b95e1125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:08.407670 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-a251b8c8-c3cf-48b4-881b-fb36b95e1125 None None] MFA Rules not processed for user `c8208192d40243e9a81afb254a9326b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:08.408874 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-a251b8c8-c3cf-48b4-881b-fb36b95e1125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:08.518941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 698/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Aug 30 11:42:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 11:42:11.812421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee60beb-8ac1-49dd-9116-e94bebbfa6fa None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:11.812987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee60beb-8ac1-49dd-9116-e94bebbfa6fa None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:11.812987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee60beb-8ac1-49dd-9116-e94bebbfa6fa None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:11.814286 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 658/1356] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:11.819592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8194c-bbe4-4066-ae72-6e95eb74c5a9 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:11.820033 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8194c-bbe4-4066-ae72-6e95eb74c5a9 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:11.820128 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8194c-bbe4-4066-ae72-6e95eb74c5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:11.857808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:11.895870 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-e4f8194c-bbe4-4066-ae72-6e95eb74c5a9 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:11.897153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-e4f8194c-bbe4-4066-ae72-6e95eb74c5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:12.034283 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 699/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:12.329822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd1bfb-64c6-49e8-acf6-3e3a85fe4735 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:12.330067 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd1bfb-64c6-49e8-acf6-3e3a85fe4735 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:12.330530 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd1bfb-64c6-49e8-acf6-3e3a85fe4735 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:12.331900 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 659/1358] 10.222.0.16 () {54 vars in 962 bytes} [Tue Aug 30 11:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:12.338450 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf28ad-d58b-4eb5-8d95-2ee259905778 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:12.339143 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf28ad-d58b-4eb5-8d95-2ee259905778 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:12.339525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf28ad-d58b-4eb5-8d95-2ee259905778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:12.432087 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-15bf28ad-d58b-4eb5-8d95-2ee259905778 None None] MFA Rules not processed for user `0bb80229c3234d5f9599b16870c69934`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:12.433531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-15bf28ad-d58b-4eb5-8d95-2ee259905778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:12.492813 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:12.560088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 700/1359] 10.222.0.16 () {58 vars in 1059 bytes} [Tue Aug 30 11:42:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:12.568028 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd1bfb-64c6-49e8-acf6-3e3a85fe4735 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:12.569770 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-c0bd1bfb-64c6-49e8-acf6-3e3a85fe4735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:12.721900 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd1bfb-64c6-49e8-acf6-3e3a85fe4735 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:12.797952 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:12.839831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a219be-53a5-424c-aa98-b9cb605fa140 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:12.840484 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a219be-53a5-424c-aa98-b9cb605fa140 service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:12.840580 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a219be-53a5-424c-aa98-b9cb605fa140 service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:12.840859 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a219be-53a5-424c-aa98-b9cb605fa140 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:12.842450 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-82a219be-53a5-424c-aa98-b9cb605fa140 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:12.932387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a219be-53a5-424c-aa98-b9cb605fa140 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:12.933991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a219be-53a5-424c-aa98-b9cb605fa140 service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:12.952501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 660/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 11:42:15.044168 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe5932-a0b0-4852-a68f-98b7b2ad40c8 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:15.044759 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe5932-a0b0-4852-a68f-98b7b2ad40c8 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:15.044759 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe5932-a0b0-4852-a68f-98b7b2ad40c8 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:15.046217 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 701/1361] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:15.051225 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b0ae-04dd-41f7-b08d-a189f896b21b None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:15.051626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b0ae-04dd-41f7-b08d-a189f896b21b None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:15.051834 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b0ae-04dd-41f7-b08d-a189f896b21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:15.125960 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-7ed8b0ae-04dd-41f7-b08d-a189f896b21b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:15.127102 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-7ed8b0ae-04dd-41f7-b08d-a189f896b21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:15.169626 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:15.234373 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 661/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:15.525281 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe5932-a0b0-4852-a68f-98b7b2ad40c8 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:15.526829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-98fe5932-a0b0-4852-a68f-98b7b2ad40c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:15.694261 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe5932-a0b0-4852-a68f-98b7b2ad40c8 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:15.704603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:15.813236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d14ec-118d-45e1-af4e-cabefb8fe0e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:15.813826 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d14ec-118d-45e1-af4e-cabefb8fe0e5 service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:15.814094 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d14ec-118d-45e1-af4e-cabefb8fe0e5 service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:15.814360 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d14ec-118d-45e1-af4e-cabefb8fe0e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:15.815979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-1d1d14ec-118d-45e1-af4e-cabefb8fe0e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:15.910357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d14ec-118d-45e1-af4e-cabefb8fe0e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:15.912032 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d14ec-118d-45e1-af4e-cabefb8fe0e5 service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:15.932803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 702/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:18.136026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a2374-c040-4c7c-ab93-ee181620a7d0 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:18.136948 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a2374-c040-4c7c-ab93-ee181620a7d0 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:18.137220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a2374-c040-4c7c-ab93-ee181620a7d0 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:18.138601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 662/1364] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:18.144168 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797236f7-48af-41d4-8c2e-9637f1f2bb7d None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:18.144601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797236f7-48af-41d4-8c2e-9637f1f2bb7d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:18.144694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797236f7-48af-41d4-8c2e-9637f1f2bb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:18.167858 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:18.239163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-797236f7-48af-41d4-8c2e-9637f1f2bb7d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:18.240609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-797236f7-48af-41d4-8c2e-9637f1f2bb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:18.356196 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 703/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:18.638217 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a82b4-78ca-403f-8d96-ec74b04f68f9 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:18.638786 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a82b4-78ca-403f-8d96-ec74b04f68f9 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:18.639311 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a82b4-78ca-403f-8d96-ec74b04f68f9 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:18.640876 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 663/1366] 10.222.0.16 () {54 vars in 962 bytes} [Tue Aug 30 11:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:18.646459 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70586b1-cb0a-44d5-8b88-cf3769297a82 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:18.646798 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70586b1-cb0a-44d5-8b88-cf3769297a82 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:18.647169 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70586b1-cb0a-44d5-8b88-cf3769297a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:18.730370 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-c70586b1-cb0a-44d5-8b88-cf3769297a82 None None] MFA Rules not processed for user `0bb80229c3234d5f9599b16870c69934`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:18.731592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c70586b1-cb0a-44d5-8b88-cf3769297a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:18.768251 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:18.854017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 704/1367] 10.222.0.16 () {58 vars in 1059 bytes} [Tue Aug 30 11:42:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:18.862325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a82b4-78ca-403f-8d96-ec74b04f68f9 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:18.864091 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-295a82b4-78ca-403f-8d96-ec74b04f68f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:19.023821 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a82b4-78ca-403f-8d96-ec74b04f68f9 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:19.045459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:19.143844 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbe12d-bfb6-4ecd-8e4d-5060167b71d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:19.144557 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbe12d-bfb6-4ecd-8e4d-5060167b71d0 service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:19.144841 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbe12d-bfb6-4ecd-8e4d-5060167b71d0 service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:19.144966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbe12d-bfb6-4ecd-8e4d-5060167b71d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:19.146659 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-57bbe12d-bfb6-4ecd-8e4d-5060167b71d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:19.242737 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbe12d-bfb6-4ecd-8e4d-5060167b71d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:19.244414 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbe12d-bfb6-4ecd-8e4d-5060167b71d0 service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:19.264920 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 664/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 11:42:19.974277 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f289106-4420-420f-b136-5a65b2f424f1 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:19.974836 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f289106-4420-420f-b136-5a65b2f424f1 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:19.974836 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f289106-4420-420f-b136-5a65b2f424f1 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:19.976353 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 705/1369] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:19.981496 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8ee51-e56b-46a0-96b5-133a96dd71b4 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:19.982254 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8ee51-e56b-46a0-96b5-133a96dd71b4 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:19.982530 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8ee51-e56b-46a0-96b5-133a96dd71b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:20.018022 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:20.053855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-eae8ee51-e56b-46a0-96b5-133a96dd71b4 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:20.055063 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-eae8ee51-e56b-46a0-96b5-133a96dd71b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:20.155171 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 665/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:20.731203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463177-ab90-4ad1-a1e9-8eb30fc45641 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:20.731598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463177-ab90-4ad1-a1e9-8eb30fc45641 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:20.731962 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463177-ab90-4ad1-a1e9-8eb30fc45641 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:20.733495 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 706/1371] 10.222.0.16 () {54 vars in 964 bytes} [Tue Aug 30 11:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:20.738480 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2b3f3-6f4d-4566-897c-431bb8056e4d None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:20.738777 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2b3f3-6f4d-4566-897c-431bb8056e4d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:20.739002 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2b3f3-6f4d-4566-897c-431bb8056e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:20.822084 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-78c2b3f3-6f4d-4566-897c-431bb8056e4d None None] MFA Rules not processed for user `d3250e8f23c64f26b4e862090f7f2e18`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:20.823528 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-78c2b3f3-6f4d-4566-897c-431bb8056e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:20.871647 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:20.904065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 666/1372] 10.222.0.16 () {58 vars in 1061 bytes} [Tue Aug 30 11:42:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:20.912061 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11463177-ab90-4ad1-a1e9-8eb30fc45641 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:20.913765 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-11463177-ab90-4ad1-a1e9-8eb30fc45641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:21.028717 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11463177-ab90-4ad1-a1e9-8eb30fc45641 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:21.070209 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:21.110669 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3250e8f23c64f26b4e862090f7f2e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:21.111466 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:21.111842 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:21.112198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:21.113977 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:21.226230 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3cc0-ecf8-4cc8-ad3a-d649bd755306 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:21.226738 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3cc0-ecf8-4cc8-ad3a-d649bd755306 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:21.227300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3cc0-ecf8-4cc8-ad3a-d649bd755306 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:21.229220 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 667/1373] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:42:21.234120 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600671f1-dbef-4363-8a44-8ddc54a9b87e None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:21.234480 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600671f1-dbef-4363-8a44-8ddc54a9b87e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:21.234932 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600671f1-dbef-4363-8a44-8ddc54a9b87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:21.247073 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:21.248937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:21.273374 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 707/1374] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:21.312175 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-600671f1-dbef-4363-8a44-8ddc54a9b87e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:21.313766 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-600671f1-dbef-4363-8a44-8ddc54a9b87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:21.432813 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 668/1375] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 11:42:21.720157 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:21.721350 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:21.790207 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:21.823691 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddc7e9-9765-41e5-ac32-f64d562f136d service cinder] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:21.941974 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36be91c-c03c-41b2-8ce9-201693940223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:21.942533 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36be91c-c03c-41b2-8ce9-201693940223 service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:21.943101 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36be91c-c03c-41b2-8ce9-201693940223 service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:21.943390 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36be91c-c03c-41b2-8ce9-201693940223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:21.944979 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f36be91c-c03c-41b2-8ce9-201693940223 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:22.030447 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:22.041676 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36be91c-c03c-41b2-8ce9-201693940223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:22.043347 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36be91c-c03c-41b2-8ce9-201693940223 service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:22.060861 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 708/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:23.962461 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b172bcb-9eae-4161-af34-b333b36203db None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:23.963010 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b172bcb-9eae-4161-af34-b333b36203db None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:23.963101 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b172bcb-9eae-4161-af34-b333b36203db None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:23.964490 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 669/1377] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:23.970163 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29af106-f2ab-46c0-838f-5bdff2eeda04 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:23.970561 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29af106-f2ab-46c0-838f-5bdff2eeda04 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:23.970679 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29af106-f2ab-46c0-838f-5bdff2eeda04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:24.037174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-b29af106-f2ab-46c0-838f-5bdff2eeda04 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:24.038371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-b29af106-f2ab-46c0-838f-5bdff2eeda04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:24.140081 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 709/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:24.424244 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b172bcb-9eae-4161-af34-b333b36203db None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:24.425770 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-2b172bcb-9eae-4161-af34-b333b36203db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:24.484419 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:24.583039 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b172bcb-9eae-4161-af34-b333b36203db None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:24.716164 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a036c-c666-47e8-aa3b-126fa4280769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:24.716871 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a036c-c666-47e8-aa3b-126fa4280769 service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:24.717243 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a036c-c666-47e8-aa3b-126fa4280769 service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:24.717606 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a036c-c666-47e8-aa3b-126fa4280769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:24.719993 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-8e6a036c-c666-47e8-aa3b-126fa4280769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:24.740939 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:24.830970 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a036c-c666-47e8-aa3b-126fa4280769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:24.833027 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a036c-c666-47e8-aa3b-126fa4280769 service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:24.855097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 670/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:26.778605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcad892-c22c-4a13-9c0e-bde1a79d4390 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:26.779385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcad892-c22c-4a13-9c0e-bde1a79d4390 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:26.779385 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcad892-c22c-4a13-9c0e-bde1a79d4390 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:26.783504 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 710/1380] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:26.786089 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4d364-feea-4622-a858-8c0e159a2c2c None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:26.786534 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4d364-feea-4622-a858-8c0e159a2c2c None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:26.786614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4d364-feea-4622-a858-8c0e159a2c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:26.854953 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-4bc4d364-feea-4622-a858-8c0e159a2c2c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:26.856178 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-4bc4d364-feea-4622-a858-8c0e159a2c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:26.944864 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:26.959385 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 671/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:27.247689 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcad892-c22c-4a13-9c0e-bde1a79d4390 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:27.249612 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-bbcad892-c22c-4a13-9c0e-bde1a79d4390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:27.319891 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:27.415639 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcad892-c22c-4a13-9c0e-bde1a79d4390 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:27.548695 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718c0dd-096a-4cbb-b8f8-8b08e83f182e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:27.549495 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718c0dd-096a-4cbb-b8f8-8b08e83f182e service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:27.549775 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718c0dd-096a-4cbb-b8f8-8b08e83f182e service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:27.550044 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718c0dd-096a-4cbb-b8f8-8b08e83f182e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:27.551655 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-8718c0dd-096a-4cbb-b8f8-8b08e83f182e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:27.574971 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:27.647525 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718c0dd-096a-4cbb-b8f8-8b08e83f182e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:27.649122 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718c0dd-096a-4cbb-b8f8-8b08e83f182e service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:27.668115 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 711/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:29.575761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95060065-321f-48a2-9397-ac9410cb3403 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:29.576438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95060065-321f-48a2-9397-ac9410cb3403 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:29.576438 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95060065-321f-48a2-9397-ac9410cb3403 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:29.577684 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 672/1383] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:29.582903 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5303964-5e9b-4317-aaf1-79fd52cac84b None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:29.583387 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5303964-5e9b-4317-aaf1-79fd52cac84b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:29.583484 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5303964-5e9b-4317-aaf1-79fd52cac84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:29.654055 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-c5303964-5e9b-4317-aaf1-79fd52cac84b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:29.655288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c5303964-5e9b-4317-aaf1-79fd52cac84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:29.755614 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:29.759358 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 712/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:30.040705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95060065-321f-48a2-9397-ac9410cb3403 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:30.042225 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-95060065-321f-48a2-9397-ac9410cb3403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:30.200384 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95060065-321f-48a2-9397-ac9410cb3403 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:30.278134 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:30.336637 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177b73a-9584-411f-91ce-6fd1c9b328fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:30.337248 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177b73a-9584-411f-91ce-6fd1c9b328fb service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:30.337542 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177b73a-9584-411f-91ce-6fd1c9b328fb service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:30.337790 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177b73a-9584-411f-91ce-6fd1c9b328fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:30.339425 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-e177b73a-9584-411f-91ce-6fd1c9b328fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:30.443924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177b73a-9584-411f-91ce-6fd1c9b328fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:30.446457 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177b73a-9584-411f-91ce-6fd1c9b328fb service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:30.468692 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 673/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:32.361013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f8ae6-5dbd-4904-ad89-f3b30e857e7d None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:32.361737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f8ae6-5dbd-4904-ad89-f3b30e857e7d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:32.361737 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f8ae6-5dbd-4904-ad89-f3b30e857e7d None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:32.362941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 713/1386] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:32.368791 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec756bd-c8b9-4066-a5e2-5199673b03f3 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:32.369208 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec756bd-c8b9-4066-a5e2-5199673b03f3 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:32.369584 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec756bd-c8b9-4066-a5e2-5199673b03f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:32.451501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-0ec756bd-c8b9-4066-a5e2-5199673b03f3 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:32.453044 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-0ec756bd-c8b9-4066-a5e2-5199673b03f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:32.483726 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:32.585568 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 674/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 11:42:32.870106 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f8ae6-5dbd-4904-ad89-f3b30e857e7d None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:32.871640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ef5f8ae6-5dbd-4904-ad89-f3b30e857e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:33.023235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f8ae6-5dbd-4904-ad89-f3b30e857e7d None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:33.111509 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:33.157241 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbe084-e19e-4fb1-855b-1e39ce6f8f38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:33.157853 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbe084-e19e-4fb1-855b-1e39ce6f8f38 service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:33.158122 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbe084-e19e-4fb1-855b-1e39ce6f8f38 service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:33.158382 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbe084-e19e-4fb1-855b-1e39ce6f8f38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:33.160089 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-cabbe084-e19e-4fb1-855b-1e39ce6f8f38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:33.253845 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbe084-e19e-4fb1-855b-1e39ce6f8f38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:33.255522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbe084-e19e-4fb1-855b-1e39ce6f8f38 service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:33.273636 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 714/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:35.194159 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71854f93-740c-47d2-b8fc-db42b03ac370 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:35.194831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71854f93-740c-47d2-b8fc-db42b03ac370 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:35.194831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71854f93-740c-47d2-b8fc-db42b03ac370 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:35.196444 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 675/1389] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:35.202730 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0fdb2-05de-49f3-a621-912d2ec09b5e None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:35.203208 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0fdb2-05de-49f3-a621-912d2ec09b5e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:35.203313 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0fdb2-05de-49f3-a621-912d2ec09b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:35.270056 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-c2d0fdb2-05de-49f3-a621-912d2ec09b5e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:35.271684 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c2d0fdb2-05de-49f3-a621-912d2ec09b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:35.308668 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:35.376126 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 715/1390] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:35.659677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71854f93-740c-47d2-b8fc-db42b03ac370 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:35.661501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-71854f93-740c-47d2-b8fc-db42b03ac370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:35.844231 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71854f93-740c-47d2-b8fc-db42b03ac370 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:35.853660 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:35.983593 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a54331-3caf-4071-ba5d-cc65afb8b3fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:35.984331 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a54331-3caf-4071-ba5d-cc65afb8b3fd service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:35.984780 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a54331-3caf-4071-ba5d-cc65afb8b3fd service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:35.985091 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a54331-3caf-4071-ba5d-cc65afb8b3fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:35.988410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-21a54331-3caf-4071-ba5d-cc65afb8b3fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:36.092518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a54331-3caf-4071-ba5d-cc65afb8b3fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:36.094120 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a54331-3caf-4071-ba5d-cc65afb8b3fd service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:36.111975 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 676/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:38.029091 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b7fa2-14cc-44e4-996e-02118edefd51 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:38.029708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b7fa2-14cc-44e4-996e-02118edefd51 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:38.029708 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b7fa2-14cc-44e4-996e-02118edefd51 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:38.030967 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 716/1392] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:42:38.036248 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75e4ec-a802-4e48-b140-1b7c5d23101d None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:38.036705 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75e4ec-a802-4e48-b140-1b7c5d23101d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:38.036811 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75e4ec-a802-4e48-b140-1b7c5d23101d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:38.052393 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:38.112212 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-1d75e4ec-a802-4e48-b140-1b7c5d23101d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:38.113406 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-1d75e4ec-a802-4e48-b140-1b7c5d23101d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:38.217838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 677/1393] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 11:42:38.280996 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910157a3-e187-4870-8134-b73b1a700e92 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:38.282828 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910157a3-e187-4870-8134-b73b1a700e92 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:38.283199 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910157a3-e187-4870-8134-b73b1a700e92 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:38.284664 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 717/1394] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:38.290423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b7b1a-ab95-4415-92ff-3f9ee343db7b None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:38.290737 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b7b1a-ab95-4415-92ff-3f9ee343db7b None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:38.291035 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b7b1a-ab95-4415-92ff-3f9ee343db7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:38.364596 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-5e4b7b1a-ab95-4415-92ff-3f9ee343db7b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:38.365825 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-5e4b7b1a-ab95-4415-92ff-3f9ee343db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:38.416393 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:38.475855 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 678/1395] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:38.503394 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910157a3-e187-4870-8134-b73b1a700e92 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:38.504897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-910157a3-e187-4870-8134-b73b1a700e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:38.656919 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910157a3-e187-4870-8134-b73b1a700e92 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:38.675845 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:38.778590 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848ba5-f701-456c-8e53-23f115b30fd1 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:38.778831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848ba5-f701-456c-8e53-23f115b30fd1 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:38.779137 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848ba5-f701-456c-8e53-23f115b30fd1 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:38.780497 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 679/1396] 10.222.0.16 () {54 vars in 967 bytes} [Tue Aug 30 11:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:42:38.784344 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:38.785003 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:38.785339 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:38.785507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088cf0a-2d2c-497d-ba54-01046a1868dd None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:38.785654 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:38.785773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088cf0a-2d2c-497d-ba54-01046a1868dd None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:38.786034 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088cf0a-2d2c-497d-ba54-01046a1868dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:38.787655 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:38.868734 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-2088cf0a-2d2c-497d-ba54-01046a1868dd None None] MFA Rules not processed for user `efbe019e4d4646428bed474cef4009b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:38.870240 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-2088cf0a-2d2c-497d-ba54-01046a1868dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:38.889197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:38.890898 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:38.910781 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 718/1397] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:38.955381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 680/1398] 10.222.0.16 () {58 vars in 1064 bytes} [Tue Aug 30 11:42:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 11:42:38.963039 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:38.964392 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:39.003638 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:39.060972 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1a10e-9963-4836-aae5-0ab58c84fc87 service nova] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:39.145977 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c5dff-1907-4661-80ba-de7e808c9c93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbe019e4d4646428bed474cef4009b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:39.146581 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c5dff-1907-4661-80ba-de7e808c9c93 service glance] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:39.146829 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c5dff-1907-4661-80ba-de7e808c9c93 service glance] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:39.147100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c5dff-1907-4661-80ba-de7e808c9c93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:39.148694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ab3c5dff-1907-4661-80ba-de7e808c9c93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:39.251464 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:39.262675 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c5dff-1907-4661-80ba-de7e808c9c93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:39.264079 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c5dff-1907-4661-80ba-de7e808c9c93 service glance] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:39.282133 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 719/1399] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:40.023912 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4d2c5-0461-4bc5-8746-ee9af7bb3b59 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:40.024409 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4d2c5-0461-4bc5-8746-ee9af7bb3b59 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:40.024513 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4d2c5-0461-4bc5-8746-ee9af7bb3b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:40.076421 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:40.097166 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-63c4d2c5-0461-4bc5-8746-ee9af7bb3b59 None None] MFA Rules not processed for user `ee318396a8a04174a509a5faa8b86aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:40.098365 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-63c4d2c5-0461-4bc5-8746-ee9af7bb3b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:40.183616 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 681/1400] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Aug 30 11:42:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:40.225523 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39486271-feee-4c28-872a-e478b513ca07 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:40.226019 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39486271-feee-4c28-872a-e478b513ca07 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:40.226188 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39486271-feee-4c28-872a-e478b513ca07 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:40.227693 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 720/1401] 10.222.0.16 () {54 vars in 943 bytes} [Tue Aug 30 11:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:40.234410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9756-894b-4f42-95ee-bd36ce863405 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:40.234692 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9756-894b-4f42-95ee-bd36ce863405 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:40.234959 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9756-894b-4f42-95ee-bd36ce863405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:40.319292 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-7bca9756-894b-4f42-95ee-bd36ce863405 None None] MFA Rules not processed for user `8bbb8fb036684305bc00290c519939ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:40.320492 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-7bca9756-894b-4f42-95ee-bd36ce863405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:40.389701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:40.434615 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 682/1402] 10.222.0.16 () {58 vars in 1040 bytes} [Tue Aug 30 11:42:40 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:40.444128 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39486271-feee-4c28-872a-e478b513ca07 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:40.445643 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-39486271-feee-4c28-872a-e478b513ca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:40.590329 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39486271-feee-4c28-872a-e478b513ca07 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:40.662595 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:40.699736 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7a2b9-fb18-4f36-86f4-79560791ba7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bbb8fb036684305bc00290c519939ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:40.700361 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7a2b9-fb18-4f36-86f4-79560791ba7f service swift] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:40.700636 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7a2b9-fb18-4f36-86f4-79560791ba7f service swift] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:40.700897 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7a2b9-fb18-4f36-86f4-79560791ba7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:40.702515 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-baf7a2b9-fb18-4f36-86f4-79560791ba7f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:40.785858 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7a2b9-fb18-4f36-86f4-79560791ba7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:40.787496 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7a2b9-fb18-4f36-86f4-79560791ba7f service swift] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:40.795718 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 721/1403] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Aug 30 11:42:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 11:42:40.877603 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701af77-9ac0-481a-861e-e6d8b8bae29d None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:40.878386 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701af77-9ac0-481a-861e-e6d8b8bae29d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:40.878386 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701af77-9ac0-481a-861e-e6d8b8bae29d None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:40.879959 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 683/1404] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:40.885349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a88c3-ff52-4966-9df2-224636d346c4 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:40.885632 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a88c3-ff52-4966-9df2-224636d346c4 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:40.885911 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a88c3-ff52-4966-9df2-224636d346c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:40.963398 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-fe2a88c3-ff52-4966-9df2-224636d346c4 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:40.964627 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-fe2a88c3-ff52-4966-9df2-224636d346c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:41.025672 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:41.079531 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 722/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:41.364344 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701af77-9ac0-481a-861e-e6d8b8bae29d None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:41.365872 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-3701af77-9ac0-481a-861e-e6d8b8bae29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:41.531041 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701af77-9ac0-481a-861e-e6d8b8bae29d None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:41.579487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:41.657987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a694d3c-8e64-4df9-9488-d21d512628a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:41.658536 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a694d3c-8e64-4df9-9488-d21d512628a9 service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:41.658814 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a694d3c-8e64-4df9-9488-d21d512628a9 service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:41.659070 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a694d3c-8e64-4df9-9488-d21d512628a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:41.660677 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-6a694d3c-8e64-4df9-9488-d21d512628a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:41.753510 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a694d3c-8e64-4df9-9488-d21d512628a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:41.755190 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a694d3c-8e64-4df9-9488-d21d512628a9 service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:41.773272 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 684/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:43.700921 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070734f9-eaa6-4c7c-ab7e-8cb6e54e38f0 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:43.701543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070734f9-eaa6-4c7c-ab7e-8cb6e54e38f0 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:43.701543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070734f9-eaa6-4c7c-ab7e-8cb6e54e38f0 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:43.702943 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 723/1407] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:42:43.709094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946817c1-0ce7-426e-8a55-bbf86e514000 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:43.709094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946817c1-0ce7-426e-8a55-bbf86e514000 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:43.709094 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946817c1-0ce7-426e-8a55-bbf86e514000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:43.776173 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:43.779028 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-946817c1-0ce7-426e-8a55-bbf86e514000 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:42:43.780653 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-946817c1-0ce7-426e-8a55-bbf86e514000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:43.906171 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 685/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:42:44.197057 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070734f9-eaa6-4c7c-ab7e-8cb6e54e38f0 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:42:44.198698 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-070734f9-eaa6-4c7c-ab7e-8cb6e54e38f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:44.361584 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070734f9-eaa6-4c7c-ab7e-8cb6e54e38f0 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:42:44.402769 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:44.489475 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad3e50-9479-4c52-b4e4-a893b2271185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:42:44.490087 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad3e50-9479-4c52-b4e4-a893b2271185 service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:42:44.490365 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad3e50-9479-4c52-b4e4-a893b2271185 service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:42:44.490627 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad3e50-9479-4c52-b4e4-a893b2271185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:42:44.492259 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-63ad3e50-9479-4c52-b4e4-a893b2271185 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:42:44.589842 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad3e50-9479-4c52-b4e4-a893b2271185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:42:44.591590 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad3e50-9479-4c52-b4e4-a893b2271185 service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:42:44.610168 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 724/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:42:53.904465 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=44 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:42:54.607997 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=80 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:02.790132 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d78e2-8c6d-4093-89f0-9dc18a7df01d None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:02.790749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d78e2-8c6d-4093-89f0-9dc18a7df01d None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:02.790749 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d78e2-8c6d-4093-89f0-9dc18a7df01d None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:02.792075 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 686/1410] 10.222.0.16 () {54 vars in 918 bytes} [Tue Aug 30 11:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:02.798194 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0295eb-48da-4e49-a239-073d33e1ad84 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:02.798887 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0295eb-48da-4e49-a239-073d33e1ad84 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:02.799019 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0295eb-48da-4e49-a239-073d33e1ad84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:02.884558 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-bb0295eb-48da-4e49-a239-073d33e1ad84 None None] MFA Rules not processed for user `c8208192d40243e9a81afb254a9326b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:02.885745 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-bb0295eb-48da-4e49-a239-073d33e1ad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:03.012998 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 725/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Aug 30 11:43:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:03.029744 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f170d6-f356-47d0-a7b6-4eb92bec9360 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:03.030026 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f170d6-f356-47d0-a7b6-4eb92bec9360 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:03.030289 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f170d6-f356-47d0-a7b6-4eb92bec9360 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:03.031664 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 687/1412] 10.222.0.16 () {54 vars in 918 bytes} [Tue Aug 30 11:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:03.036671 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f4fa-4038-47a8-aff1-753a31328b67 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:03.036913 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f4fa-4038-47a8-aff1-753a31328b67 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:03.037216 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f4fa-4038-47a8-aff1-753a31328b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:03.110347 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:03.114010 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-5944f4fa-4038-47a8-aff1-753a31328b67 None None] MFA Rules not processed for user `0bb80229c3234d5f9599b16870c69934`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:03.116357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-5944f4fa-4038-47a8-aff1-753a31328b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:03.237034 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 726/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Aug 30 11:43:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 11:43:05.617018 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e9456-594c-4517-976a-a77f87a8be12 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:05.617639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e9456-594c-4517-976a-a77f87a8be12 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:05.617639 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e9456-594c-4517-976a-a77f87a8be12 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:05.618910 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 688/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:05.627147 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d59550-cc68-4961-a45e-5cc3c1ac38aa None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:05.627576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d59550-cc68-4961-a45e-5cc3c1ac38aa None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:05.627697 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d59550-cc68-4961-a45e-5cc3c1ac38aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:05.697811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-b9d59550-cc68-4961-a45e-5cc3c1ac38aa None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:05.698963 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-b9d59550-cc68-4961-a45e-5cc3c1ac38aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:05.759927 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:05.803051 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 727/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:06.088288 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e9456-594c-4517-976a-a77f87a8be12 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:06.090346 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-362e9456-594c-4517-976a-a77f87a8be12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:06.245702 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e9456-594c-4517-976a-a77f87a8be12 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:06.321365 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:06.364764 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148665be-9eaf-44db-a0e2-7549184b9dda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:06.365377 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148665be-9eaf-44db-a0e2-7549184b9dda service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:06.365649 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148665be-9eaf-44db-a0e2-7549184b9dda service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:06.365907 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148665be-9eaf-44db-a0e2-7549184b9dda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:06.367591 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-148665be-9eaf-44db-a0e2-7549184b9dda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:06.462617 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148665be-9eaf-44db-a0e2-7549184b9dda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:06.464263 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148665be-9eaf-44db-a0e2-7549184b9dda service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:06.482844 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 689/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:07.371839 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784b7a3-1a0a-4ae8-b622-3c01dc74d14c None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:07.372428 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784b7a3-1a0a-4ae8-b622-3c01dc74d14c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:07.372428 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784b7a3-1a0a-4ae8-b622-3c01dc74d14c None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:07.373726 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 728/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 11:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:07.379771 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b0f27-01d3-4626-b3f1-2b13372ea3dd None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:07.380515 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b0f27-01d3-4626-b3f1-2b13372ea3dd None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:07.380695 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b0f27-01d3-4626-b3f1-2b13372ea3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:07.466966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-209b0f27-01d3-4626-b3f1-2b13372ea3dd None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:07.468201 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-209b0f27-01d3-4626-b3f1-2b13372ea3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:07.495325 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:07.567333 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 690/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 11:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:07.724230 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784b7a3-1a0a-4ae8-b622-3c01dc74d14c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:07.726711 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-4784b7a3-1a0a-4ae8-b622-3c01dc74d14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:07.889642 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784b7a3-1a0a-4ae8-b622-3c01dc74d14c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:07.939952 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:08.020911 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3157eb-3782-413e-9ae0-319ef9a0a7a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:08.021698 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3157eb-3782-413e-9ae0-319ef9a0a7a6 service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:08.022114 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3157eb-3782-413e-9ae0-319ef9a0a7a6 service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:08.022529 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3157eb-3782-413e-9ae0-319ef9a0a7a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:08.024922 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-2c3157eb-3782-413e-9ae0-319ef9a0a7a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:08.120541 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3157eb-3782-413e-9ae0-319ef9a0a7a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:08.122182 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3157eb-3782-413e-9ae0-319ef9a0a7a6 service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:08.140945 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 729/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:09.025101 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac8b08-8d9f-4f27-be46-0741b2123d42 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:09.025915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac8b08-8d9f-4f27-be46-0741b2123d42 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:09.025915 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac8b08-8d9f-4f27-be46-0741b2123d42 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:09.028105 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 691/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 11:43:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:09.036415 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6d7fa-ab99-46d6-8bda-38a4dd88304c None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:09.037156 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6d7fa-ab99-46d6-8bda-38a4dd88304c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:09.037344 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6d7fa-ab99-46d6-8bda-38a4dd88304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:09.097513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:09.121161 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-07d6d7fa-ab99-46d6-8bda-38a4dd88304c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:09.122357 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-07d6d7fa-ab99-46d6-8bda-38a4dd88304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:09.221227 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 730/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 11:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:09.370277 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac8b08-8d9f-4f27-be46-0741b2123d42 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:09.372895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-2aac8b08-8d9f-4f27-be46-0741b2123d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:09.529453 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac8b08-8d9f-4f27-be46-0741b2123d42 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:09.542469 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:09.659329 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c104f-233e-47b9-b534-206f28d975f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:09.659858 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c104f-233e-47b9-b534-206f28d975f1 service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:09.660129 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c104f-233e-47b9-b534-206f28d975f1 service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:09.660492 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c104f-233e-47b9-b534-206f28d975f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:09.662072 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-b69c104f-233e-47b9-b534-206f28d975f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:09.754067 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c104f-233e-47b9-b534-206f28d975f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:09.755762 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c104f-233e-47b9-b534-206f28d975f1 service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:09.774454 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 692/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:10.627977 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eda00e-9a8b-46b1-9137-7e7d34347c93 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:10.628866 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eda00e-9a8b-46b1-9137-7e7d34347c93 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:10.628866 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eda00e-9a8b-46b1-9137-7e7d34347c93 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:10.630894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 731/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 11:43:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:10.637655 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d00ffb-5a17-4181-9d4e-222a98229fd6 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:10.638109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d00ffb-5a17-4181-9d4e-222a98229fd6 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:10.638206 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d00ffb-5a17-4181-9d4e-222a98229fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:10.660381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:10.720929 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-75d00ffb-5a17-4181-9d4e-222a98229fd6 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:10.722032 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-75d00ffb-5a17-4181-9d4e-222a98229fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:10.819860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 693/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 11:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:10.972054 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eda00e-9a8b-46b1-9137-7e7d34347c93 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:10.974256 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f1eda00e-9a8b-46b1-9137-7e7d34347c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:11.104095 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:11.141593 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eda00e-9a8b-46b1-9137-7e7d34347c93 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:11.258177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de018e8-4231-42f3-b07a-356cd76bbe36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:11.258820 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de018e8-4231-42f3-b07a-356cd76bbe36 service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:11.259088 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de018e8-4231-42f3-b07a-356cd76bbe36 service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:11.259379 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de018e8-4231-42f3-b07a-356cd76bbe36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:11.260951 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-2de018e8-4231-42f3-b07a-356cd76bbe36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:11.339919 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:11.357189 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de018e8-4231-42f3-b07a-356cd76bbe36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:11.358851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de018e8-4231-42f3-b07a-356cd76bbe36 service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:11.377886 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 732/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:12.240657 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b2e43-cb35-4f76-9a93-89f446dc886e None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:12.243006 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b2e43-cb35-4f76-9a93-89f446dc886e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:12.243006 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b2e43-cb35-4f76-9a93-89f446dc886e None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:12.243006 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 694/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 11:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:12.248431 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf7cf6-5c10-4b10-a17b-47706ea6b85c None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:12.249158 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf7cf6-5c10-4b10-a17b-47706ea6b85c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:12.249356 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf7cf6-5c10-4b10-a17b-47706ea6b85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:12.331076 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-b9bf7cf6-5c10-4b10-a17b-47706ea6b85c None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:12.332308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-b9bf7cf6-5c10-4b10-a17b-47706ea6b85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:12.432704 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 733/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 11:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 11:43:12.580226 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b2e43-cb35-4f76-9a93-89f446dc886e None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:12.581648 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-e13b2e43-cb35-4f76-9a93-89f446dc886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:12.668322 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:12.730782 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b2e43-cb35-4f76-9a93-89f446dc886e None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:12.854124 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e5a20-4c59-4be1-858d-674efa1273f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:12.855229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e5a20-4c59-4be1-858d-674efa1273f9 service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:12.855506 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e5a20-4c59-4be1-858d-674efa1273f9 service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:12.855759 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e5a20-4c59-4be1-858d-674efa1273f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:12.857366 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-cb7e5a20-4c59-4be1-858d-674efa1273f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:12.908460 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:12.951228 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e5a20-4c59-4be1-858d-674efa1273f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:12.952823 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e5a20-4c59-4be1-858d-674efa1273f9 service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:12.977987 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 695/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:13.799681 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf6193-43fc-47a4-b964-50f83282b9a2 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:13.800275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf6193-43fc-47a4-b964-50f83282b9a2 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:13.800275 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf6193-43fc-47a4-b964-50f83282b9a2 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:13.802232 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 734/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 11:43:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:13.810361 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823cad3-358e-4e88-82d8-f7389463aea9 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:13.810773 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823cad3-358e-4e88-82d8-f7389463aea9 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:13.810931 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823cad3-358e-4e88-82d8-f7389463aea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:13.900157 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-6823cad3-358e-4e88-82d8-f7389463aea9 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:13.901377 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-6823cad3-358e-4e88-82d8-f7389463aea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:14.008917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 696/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 11:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:14.162410 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf6193-43fc-47a4-b964-50f83282b9a2 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:14.164780 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-a4bf6193-43fc-47a4-b964-50f83282b9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:14.220980 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:14.323611 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf6193-43fc-47a4-b964-50f83282b9a2 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:14.442580 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c41d25-d95d-47c1-822c-5b9c1a14db3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:14.443239 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c41d25-d95d-47c1-822c-5b9c1a14db3a service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:14.443513 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c41d25-d95d-47c1-822c-5b9c1a14db3a service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:14.443778 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c41d25-d95d-47c1-822c-5b9c1a14db3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:14.445375 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f7c41d25-d95d-47c1-822c-5b9c1a14db3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:14.453198 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:14.542271 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c41d25-d95d-47c1-822c-5b9c1a14db3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:14.543940 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c41d25-d95d-47c1-822c-5b9c1a14db3a service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:14.562312 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 735/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:15.364824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff6c6b-d610-4568-bf19-b354b165e17a None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:15.365456 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff6c6b-d610-4568-bf19-b354b165e17a None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:15.365456 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff6c6b-d610-4568-bf19-b354b165e17a None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:15.366750 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 697/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 11:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:15.372916 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e3d14-38e8-484e-b113-10512f82718e None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:15.373386 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e3d14-38e8-484e-b113-10512f82718e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:15.373492 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e3d14-38e8-484e-b113-10512f82718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:15.456204 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-d51e3d14-38e8-484e-b113-10512f82718e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:15.457407 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-d51e3d14-38e8-484e-b113-10512f82718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:15.534220 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:15.564678 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 736/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 11:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:15.727888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff6c6b-d610-4568-bf19-b354b165e17a None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:15.730323 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-f0ff6c6b-d610-4568-bf19-b354b165e17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:15.752818 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:15.922852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff6c6b-d610-4568-bf19-b354b165e17a None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:16.018416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:16.048021 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41662d-68a4-412e-8034-068c41781e62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:16.048632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41662d-68a4-412e-8034-068c41781e62 service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:16.048902 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41662d-68a4-412e-8034-068c41781e62 service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:16.049164 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41662d-68a4-412e-8034-068c41781e62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:16.050784 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-4e41662d-68a4-412e-8034-068c41781e62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:16.155315 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41662d-68a4-412e-8034-068c41781e62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:16.156922 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41662d-68a4-412e-8034-068c41781e62 service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:16.175619 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 698/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:16.984140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae93b6-abbd-4355-9f95-dd20dd8deb13 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:16.984760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae93b6-abbd-4355-9f95-dd20dd8deb13 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:16.984760 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae93b6-abbd-4355-9f95-dd20dd8deb13 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:16.986052 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 737/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 11:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:16.993020 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5d5cf-9609-413c-a71e-3765024929da None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:16.993492 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5d5cf-9609-413c-a71e-3765024929da None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:16.993760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5d5cf-9609-413c-a71e-3765024929da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:17.077300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-8bf5d5cf-9609-413c-a71e-3765024929da None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:17.078574 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-8bf5d5cf-9609-413c-a71e-3765024929da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:17.130151 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:17.188296 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 699/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 11:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:17.341616 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae93b6-abbd-4355-9f95-dd20dd8deb13 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:17.343976 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c6ae93b6-abbd-4355-9f95-dd20dd8deb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:17.499822 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae93b6-abbd-4355-9f95-dd20dd8deb13 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:17.555892 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:17.618637 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db89109-c50d-46f3-a3e8-1d882d738aef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:17.619203 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db89109-c50d-46f3-a3e8-1d882d738aef service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:17.619472 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db89109-c50d-46f3-a3e8-1d882d738aef service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:17.619731 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db89109-c50d-46f3-a3e8-1d882d738aef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:17.621307 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-9db89109-c50d-46f3-a3e8-1d882d738aef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:17.721214 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db89109-c50d-46f3-a3e8-1d882d738aef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:17.722815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db89109-c50d-46f3-a3e8-1d882d738aef service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:17.740976 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 738/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:18.555629 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41475a-3e3b-4999-91d2-9f2a8a4d4a1f None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:18.556487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41475a-3e3b-4999-91d2-9f2a8a4d4a1f None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:18.556487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41475a-3e3b-4999-91d2-9f2a8a4d4a1f None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:18.558545 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 700/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 11:43:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:18.564668 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed020fe-08cb-44f9-adc4-90de5e359221 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:18.565432 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed020fe-08cb-44f9-adc4-90de5e359221 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:18.565609 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed020fe-08cb-44f9-adc4-90de5e359221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:18.652768 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-eed020fe-08cb-44f9-adc4-90de5e359221 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:18.654267 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-eed020fe-08cb-44f9-adc4-90de5e359221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:18.656449 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:18.760711 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 739/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 11:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:18.919495 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41475a-3e3b-4999-91d2-9f2a8a4d4a1f None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:18.921582 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-ba41475a-3e3b-4999-91d2-9f2a8a4d4a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:19.092976 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41475a-3e3b-4999-91d2-9f2a8a4d4a1f None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:19.120920 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:19.218432 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b07432-4cdb-4cb8-bc7e-e17ccbd4e91c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:19.218939 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b07432-4cdb-4cb8-bc7e-e17ccbd4e91c service nova] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:19.219251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b07432-4cdb-4cb8-bc7e-e17ccbd4e91c service nova] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:19.219505 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b07432-4cdb-4cb8-bc7e-e17ccbd4e91c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:19.221109 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-d5b07432-4cdb-4cb8-bc7e-e17ccbd4e91c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:19.325620 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b07432-4cdb-4cb8-bc7e-e17ccbd4e91c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:19.327326 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b07432-4cdb-4cb8-bc7e-e17ccbd4e91c service nova] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:19.347830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 701/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:21.173588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d0764-8830-42a1-8bdb-bbd1d421c3c0 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:21.174250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d0764-8830-42a1-8bdb-bbd1d421c3c0 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:21.174250 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d0764-8830-42a1-8bdb-bbd1d421c3c0 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:21.175522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 740/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 11:43:21.182265 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e501e33-01c7-464d-8f6c-765d087b7624 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:21.183040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e501e33-01c7-464d-8f6c-765d087b7624 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:21.183258 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e501e33-01c7-464d-8f6c-765d087b7624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:21.231960 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:21.258047 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-7e501e33-01c7-464d-8f6c-765d087b7624 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:21.259316 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-7e501e33-01c7-464d-8f6c-765d087b7624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:21.365241 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 702/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:21.648833 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d0764-8830-42a1-8bdb-bbd1d421c3c0 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:21.650405 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ff3d0764-8830-42a1-8bdb-bbd1d421c3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:21.761043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d0764-8830-42a1-8bdb-bbd1d421c3c0 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:21.798661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:21.844256 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4318f03-a675-4955-b9c1-4bcb3bbff0c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:21.844859 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4318f03-a675-4955-b9c1-4bcb3bbff0c5 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:21.845126 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4318f03-a675-4955-b9c1-4bcb3bbff0c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:21.845390 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4318f03-a675-4955-b9c1-4bcb3bbff0c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:21.846954 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f4318f03-a675-4955-b9c1-4bcb3bbff0c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:21.968822 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4318f03-a675-4955-b9c1-4bcb3bbff0c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:21.970274 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4318f03-a675-4955-b9c1-4bcb3bbff0c5 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:21.989852 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 741/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:24.026864 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e261b-c8f5-4391-a259-d5e7f7091da4 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:24.027428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e261b-c8f5-4391-a259-d5e7f7091da4 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:24.027428 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e261b-c8f5-4391-a259-d5e7f7091da4 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:24.028798 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 703/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:24.035104 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c30a49-c574-4436-876e-23c3cb81889b None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:24.035593 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c30a49-c574-4436-876e-23c3cb81889b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:24.035705 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c30a49-c574-4436-876e-23c3cb81889b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:24.106294 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-42c30a49-c574-4436-876e-23c3cb81889b None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:24.107514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-42c30a49-c574-4436-876e-23c3cb81889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:24.120486 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:24.211015 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 742/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:24.490191 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e261b-c8f5-4391-a259-d5e7f7091da4 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:24.491795 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-da0e261b-c8f5-4391-a259-d5e7f7091da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:24.600501 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e261b-c8f5-4391-a259-d5e7f7091da4 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:24.613155 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:24.681756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb79ba4-602d-474f-ad9f-00f9d005f514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:24.682355 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb79ba4-602d-474f-ad9f-00f9d005f514 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:24.682625 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb79ba4-602d-474f-ad9f-00f9d005f514 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:24.682895 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb79ba4-602d-474f-ad9f-00f9d005f514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:24.684504 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-aeb79ba4-602d-474f-ad9f-00f9d005f514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:24.808783 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb79ba4-602d-474f-ad9f-00f9d005f514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:24.810145 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb79ba4-602d-474f-ad9f-00f9d005f514 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:24.830229 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 704/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:27.112308 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fd5c3-acda-4771-a51c-5928e2cc8801 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:27.113240 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fd5c3-acda-4771-a51c-5928e2cc8801 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:27.113240 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fd5c3-acda-4771-a51c-5928e2cc8801 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:27.115473 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 743/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:27.120946 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05685a9e-d0c4-4615-9929-73af67358d6f None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:27.121411 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05685a9e-d0c4-4615-9929-73af67358d6f None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:27.121522 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05685a9e-d0c4-4615-9929-73af67358d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:27.170055 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:27.189507 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-05685a9e-d0c4-4615-9929-73af67358d6f None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:27.190700 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-05685a9e-d0c4-4615-9929-73af67358d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:27.288614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 705/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:27.568808 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fd5c3-acda-4771-a51c-5928e2cc8801 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:27.571268 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f26fd5c3-acda-4771-a51c-5928e2cc8801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:27.685087 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fd5c3-acda-4771-a51c-5928e2cc8801 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:27.730005 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:27.768120 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498a48a-5c4f-49a3-bdb5-60c5be2f7800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:27.768726 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a48a-5c4f-49a3-bdb5-60c5be2f7800 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:27.768992 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a48a-5c4f-49a3-bdb5-60c5be2f7800 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:27.769254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a48a-5c4f-49a3-bdb5-60c5be2f7800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:27.770817 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-3498a48a-5c4f-49a3-bdb5-60c5be2f7800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:27.901139 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498a48a-5c4f-49a3-bdb5-60c5be2f7800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:27.902590 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498a48a-5c4f-49a3-bdb5-60c5be2f7800 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:27.921096 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 744/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:29.941416 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46854f96-bef6-4a8a-b141-e9f0d92fdccb None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:29.941990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46854f96-bef6-4a8a-b141-e9f0d92fdccb None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:29.941990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46854f96-bef6-4a8a-b141-e9f0d92fdccb None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:29.943354 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 706/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:29.950131 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb040089-730f-4c4e-b024-90028a04d67d None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:29.950909 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb040089-730f-4c4e-b024-90028a04d67d None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:29.951095 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb040089-730f-4c4e-b024-90028a04d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:30.028589 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-cb040089-730f-4c4e-b024-90028a04d67d None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:30.029811 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-cb040089-730f-4c4e-b024-90028a04d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:30.052049 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:30.139540 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 745/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:30.424539 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46854f96-bef6-4a8a-b141-e9f0d92fdccb None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:30.426831 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-46854f96-bef6-4a8a-b141-e9f0d92fdccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:30.546300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46854f96-bef6-4a8a-b141-e9f0d92fdccb None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:30.567328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:30.623884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b257e3-df02-431a-bd02-1157505ef3a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:30.624492 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b257e3-df02-431a-bd02-1157505ef3a0 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:30.624769 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b257e3-df02-431a-bd02-1157505ef3a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:30.625037 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b257e3-df02-431a-bd02-1157505ef3a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:30.626632 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-30b257e3-df02-431a-bd02-1157505ef3a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:30.745130 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b257e3-df02-431a-bd02-1157505ef3a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:30.746560 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b257e3-df02-431a-bd02-1157505ef3a0 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:30.765737 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 707/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:32.783709 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c8ab-bc33-4125-a944-906963d21cb2 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:32.784605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c8ab-bc33-4125-a944-906963d21cb2 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:32.784605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c8ab-bc33-4125-a944-906963d21cb2 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:32.786764 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 746/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:32.792440 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7a6f5-be54-4ba2-8905-864d85a03251 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:32.792889 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7a6f5-be54-4ba2-8905-864d85a03251 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:32.792995 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7a6f5-be54-4ba2-8905-864d85a03251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:32.858882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:32.861985 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-d3e7a6f5-be54-4ba2-8905-864d85a03251 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:32.863838 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-d3e7a6f5-be54-4ba2-8905-864d85a03251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:32.963709 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 708/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:33.244474 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3c8ab-bc33-4125-a944-906963d21cb2 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:33.246894 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-6dd3c8ab-bc33-4125-a944-906963d21cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:33.363600 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3c8ab-bc33-4125-a944-906963d21cb2 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:33.412960 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:33.442603 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc6e4e-d8c8-4567-bd03-fcc5c6aee035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:33.443254 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc6e4e-d8c8-4567-bd03-fcc5c6aee035 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:33.443532 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc6e4e-d8c8-4567-bd03-fcc5c6aee035 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:33.443794 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc6e4e-d8c8-4567-bd03-fcc5c6aee035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:33.445375 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-75fc6e4e-d8c8-4567-bd03-fcc5c6aee035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:33.563196 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc6e4e-d8c8-4567-bd03-fcc5c6aee035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:33.564567 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc6e4e-d8c8-4567-bd03-fcc5c6aee035 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:33.584072 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 747/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:35.537360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9570c38-5653-4fc7-93f1-9077d3d59331 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:35.537934 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9570c38-5653-4fc7-93f1-9077d3d59331 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:35.537934 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9570c38-5653-4fc7-93f1-9077d3d59331 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:35.539423 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 709/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:35.545754 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c58a7-9b66-48af-90c3-0e550133aee2 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:35.546174 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c58a7-9b66-48af-90c3-0e550133aee2 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:35.546304 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c58a7-9b66-48af-90c3-0e550133aee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:35.623588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-cd3c58a7-9b66-48af-90c3-0e550133aee2 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:35.624787 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-cd3c58a7-9b66-48af-90c3-0e550133aee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:35.654543 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:35.725751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 748/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:36.012042 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9570c38-5653-4fc7-93f1-9077d3d59331 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:36.014410 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-b9570c38-5653-4fc7-93f1-9077d3d59331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:36.133551 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9570c38-5653-4fc7-93f1-9077d3d59331 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:36.162390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:36.217740 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4847214-9b56-423d-a54a-c277aaf30b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:36.218341 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4847214-9b56-423d-a54a-c277aaf30b6f service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:36.218609 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4847214-9b56-423d-a54a-c277aaf30b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:36.218875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4847214-9b56-423d-a54a-c277aaf30b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:36.220500 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-e4847214-9b56-423d-a54a-c277aaf30b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:36.330969 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4847214-9b56-423d-a54a-c277aaf30b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:36.332381 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4847214-9b56-423d-a54a-c277aaf30b6f service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:36.350414 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 710/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:38.299741 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41463e-8088-4b77-9e8b-f78ad5c3c53b None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:38.300640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41463e-8088-4b77-9e8b-f78ad5c3c53b None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:38.300640 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41463e-8088-4b77-9e8b-f78ad5c3c53b None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:38.302645 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 749/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:38.310651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599060c-a8f5-4d3d-b34f-e805455dabf8 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:38.311097 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599060c-a8f5-4d3d-b34f-e805455dabf8 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:38.311251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599060c-a8f5-4d3d-b34f-e805455dabf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:38.380833 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-f599060c-a8f5-4d3d-b34f-e805455dabf8 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:38.382058 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-f599060c-a8f5-4d3d-b34f-e805455dabf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:38.390485 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:38.486706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 711/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 11:43:38.773177 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41463e-8088-4b77-9e8b-f78ad5c3c53b None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:38.774759 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-2c41463e-8088-4b77-9e8b-f78ad5c3c53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:38.899832 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41463e-8088-4b77-9e8b-f78ad5c3c53b None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:38.958230 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:38.977869 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed1cc1-0ee1-4d1f-90d7-9f596843efed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:38.978468 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed1cc1-0ee1-4d1f-90d7-9f596843efed service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:38.978742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed1cc1-0ee1-4d1f-90d7-9f596843efed service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:38.979007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed1cc1-0ee1-4d1f-90d7-9f596843efed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:38.980620 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-67ed1cc1-0ee1-4d1f-90d7-9f596843efed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:39.099554 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed1cc1-0ee1-4d1f-90d7-9f596843efed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:39.100933 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed1cc1-0ee1-4d1f-90d7-9f596843efed service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:39.119576 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 750/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:41.072753 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf78575-7cc5-453d-be5b-8ba002a78b86 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:41.073312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf78575-7cc5-453d-be5b-8ba002a78b86 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:41.073312 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf78575-7cc5-453d-be5b-8ba002a78b86 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:41.074667 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 712/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:41.081751 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53cb9a-0000-4efd-a648-ef971fd43ef1 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:41.082472 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53cb9a-0000-4efd-a648-ef971fd43ef1 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:41.082648 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53cb9a-0000-4efd-a648-ef971fd43ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:41.160903 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-8e53cb9a-0000-4efd-a648-ef971fd43ef1 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:41.162124 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-8e53cb9a-0000-4efd-a648-ef971fd43ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:41.202580 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:41.263763 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 751/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:41.556796 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf78575-7cc5-453d-be5b-8ba002a78b86 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:41.558370 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-2bf78575-7cc5-453d-be5b-8ba002a78b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:41.675824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf78575-7cc5-453d-be5b-8ba002a78b86 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:41.716601 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:41.754078 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30629672-32fd-4f8d-8a94-7791308592b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:41.754675 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30629672-32fd-4f8d-8a94-7791308592b4 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:41.754945 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30629672-32fd-4f8d-8a94-7791308592b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:41.755237 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30629672-32fd-4f8d-8a94-7791308592b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:41.756837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-30629672-32fd-4f8d-8a94-7791308592b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:41.884739 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30629672-32fd-4f8d-8a94-7791308592b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:41.887485 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30629672-32fd-4f8d-8a94-7791308592b4 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:41.910550 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 713/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:43.845342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc61478-ecad-4f1a-9660-9cc5e9c559e5 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:43.846236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc61478-ecad-4f1a-9660-9cc5e9c559e5 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:43.846236 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc61478-ecad-4f1a-9660-9cc5e9c559e5 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:43.848317 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 752/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:43.855944 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521b258-6029-43ff-8592-0c8c39095d9f None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:43.856390 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521b258-6029-43ff-8592-0c8c39095d9f None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:43.856487 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521b258-6029-43ff-8592-0c8c39095d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:43.937848 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-b521b258-6029-43ff-8592-0c8c39095d9f None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:43.939875 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-b521b258-6029-43ff-8592-0c8c39095d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:43.963878 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:44.042459 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 714/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:44.325961 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc61478-ecad-4f1a-9660-9cc5e9c559e5 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:44.327534 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-9dc61478-ecad-4f1a-9660-9cc5e9c559e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:44.445566 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc61478-ecad-4f1a-9660-9cc5e9c559e5 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:44.518495 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:44.526235 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e80fb3-c8b8-4aa6-b61b-3e9eeff04db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:44.526873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e80fb3-c8b8-4aa6-b61b-3e9eeff04db5 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:44.527166 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e80fb3-c8b8-4aa6-b61b-3e9eeff04db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:44.527454 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e80fb3-c8b8-4aa6-b61b-3e9eeff04db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:44.529021 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-c8e80fb3-c8b8-4aa6-b61b-3e9eeff04db5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:44.658211 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e80fb3-c8b8-4aa6-b61b-3e9eeff04db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:44.659606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e80fb3-c8b8-4aa6-b61b-3e9eeff04db5 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:44.677817 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 753/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:46.570619 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a52a3-ae95-4a5e-815f-6b0503c91174 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:46.571246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a52a3-ae95-4a5e-815f-6b0503c91174 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:46.571246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a52a3-ae95-4a5e-815f-6b0503c91174 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:46.572758 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 715/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 11:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:43:46.578784 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28d85a-ae9f-48f5-831c-f14a2435ab72 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:46.579280 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28d85a-ae9f-48f5-831c-f14a2435ab72 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:46.579384 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28d85a-ae9f-48f5-831c-f14a2435ab72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:46.653456 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-7c28d85a-ae9f-48f5-831c-f14a2435ab72 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:43:46.654661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-7c28d85a-ae9f-48f5-831c-f14a2435ab72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:46.715514 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:46.765215 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 754/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 11:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:43:47.051979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a52a3-ae95-4a5e-815f-6b0503c91174 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:43:47.054437 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-0a9a52a3-ae95-4a5e-815f-6b0503c91174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:47.177619 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a52a3-ae95-4a5e-815f-6b0503c91174 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:43:47.228278 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:47.257528 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20cf38-12c4-44b1-bcd1-96352b9066b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:43:47.258172 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20cf38-12c4-44b1-bcd1-96352b9066b0 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:43:47.258444 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20cf38-12c4-44b1-bcd1-96352b9066b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:43:47.258720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20cf38-12c4-44b1-bcd1-96352b9066b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:43:47.260353 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-df20cf38-12c4-44b1-bcd1-96352b9066b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:43:47.371486 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20cf38-12c4-44b1-bcd1-96352b9066b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:43:47.372888 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20cf38-12c4-44b1-bcd1-96352b9066b0 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:43:47.392119 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 716/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:43:56.763371 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=22 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:43:57.389729 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=66 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:06.652906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0d44f-7cf1-4672-835e-3eb964c5da51 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:06.653802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0d44f-7cf1-4672-835e-3eb964c5da51 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:06.653802 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0d44f-7cf1-4672-835e-3eb964c5da51 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:06.655961 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 755/1471] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:44:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:44:06.661372 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320632c-b57b-4e1f-ac45-243e2ea66be8 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:06.661810 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320632c-b57b-4e1f-ac45-243e2ea66be8 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:06.661917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320632c-b57b-4e1f-ac45-243e2ea66be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:06.734519 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-8320632c-b57b-4e1f-ac45-243e2ea66be8 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:44:06.735721 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-8320632c-b57b-4e1f-ac45-243e2ea66be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:06.835375 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 717/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:44:07.137100 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208d78f-3f43-44f2-bf7c-fe3e9b9fba52 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:07.137381 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208d78f-3f43-44f2-bf7c-fe3e9b9fba52 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:07.137646 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208d78f-3f43-44f2-bf7c-fe3e9b9fba52 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:07.139674 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 756/1473] 10.222.0.16 () {54 vars in 967 bytes} [Tue Aug 30 11:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:44:07.145432 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba0c80-52d0-48c7-a3d9-cc04e8e2777e None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:07.145713 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba0c80-52d0-48c7-a3d9-cc04e8e2777e None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:07.145980 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba0c80-52d0-48c7-a3d9-cc04e8e2777e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:07.228376 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-0aba0c80-52d0-48c7-a3d9-cc04e8e2777e None None] MFA Rules not processed for user `efbe019e4d4646428bed474cef4009b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:44:07.229581 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-0aba0c80-52d0-48c7-a3d9-cc04e8e2777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:07.250569 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:07.308024 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 718/1474] 10.222.0.16 () {58 vars in 1064 bytes} [Tue Aug 30 11:44:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:44:07.316159 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208d78f-3f43-44f2-bf7c-fe3e9b9fba52 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:44:07.318396 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-7208d78f-3f43-44f2-bf7c-fe3e9b9fba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:07.443990 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208d78f-3f43-44f2-bf7c-fe3e9b9fba52 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:44:07.523933 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a94916-8bde-4633-a82f-35712e3c78a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbe019e4d4646428bed474cef4009b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:44:07.524536 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a94916-8bde-4633-a82f-35712e3c78a6 service glance] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:07.524807 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a94916-8bde-4633-a82f-35712e3c78a6 service glance] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:07.525068 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a94916-8bde-4633-a82f-35712e3c78a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:07.526642 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-01a94916-8bde-4633-a82f-35712e3c78a6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:07.583588 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:07.640552 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a94916-8bde-4633-a82f-35712e3c78a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:44:07.642013 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a94916-8bde-4633-a82f-35712e3c78a6 service glance] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:44:07.660286 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 757/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:44:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:44:09.647318 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ba1c8-9b7c-436a-a1d8-5dd4cd141fc9 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:09.647956 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ba1c8-9b7c-436a-a1d8-5dd4cd141fc9 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:09.648058 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ba1c8-9b7c-436a-a1d8-5dd4cd141fc9 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:09.649445 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 719/1476] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:44:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:44:09.656179 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd64205-c73e-45db-9a16-e1fceb251493 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:09.656605 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd64205-c73e-45db-9a16-e1fceb251493 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:09.656757 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd64205-c73e-45db-9a16-e1fceb251493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:09.732098 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-9fd64205-c73e-45db-9a16-e1fceb251493 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:44:09.733280 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-9fd64205-c73e-45db-9a16-e1fceb251493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:09.846142 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 758/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:44:10.123051 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ba1c8-9b7c-436a-a1d8-5dd4cd141fc9 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:44:10.124633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-573ba1c8-9b7c-436a-a1d8-5dd4cd141fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:10.271718 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ba1c8-9b7c-436a-a1d8-5dd4cd141fc9 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:44:10.329508 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:10.380087 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492c22e-283e-4566-bce9-3aeef68b202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:44:10.380691 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492c22e-283e-4566-bce9-3aeef68b202c admin admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:10.380963 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492c22e-283e-4566-bce9-3aeef68b202c admin admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:10.381224 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492c22e-283e-4566-bce9-3aeef68b202c admin admin] PATH_INFO: `/v3/projects` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:10.381861 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492c22e-283e-4566-bce9-3aeef68b202c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:44:10.383720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: WARNING py.warnings [None req-2492c22e-283e-4566-bce9-3aeef68b202c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 11:44:10.383720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: warnings.warn(msg) Aug 30 11:44:10.383720 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: Aug 30 11:44:10.384868 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492c22e-283e-4566-bce9-3aeef68b202c admin admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:44:10.404124 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 720/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Aug 30 11:44:10 2022] GET /identity/v3/projects => generated 2553 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 11:44:12.307224 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e4cf4-23d5-416f-825e-89cb852bcb78 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:12.307815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e4cf4-23d5-416f-825e-89cb852bcb78 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:12.307815 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e4cf4-23d5-416f-825e-89cb852bcb78 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:12.309034 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 759/1479] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:44:12.315455 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf86671-78c7-4bc4-a294-2e56163bbe48 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:12.316190 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf86671-78c7-4bc4-a294-2e56163bbe48 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:12.316328 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf86671-78c7-4bc4-a294-2e56163bbe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:12.396822 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-1bf86671-78c7-4bc4-a294-2e56163bbe48 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:44:12.398114 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-1bf86671-78c7-4bc4-a294-2e56163bbe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:12.499701 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 721/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:44:12.785012 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e4cf4-23d5-416f-825e-89cb852bcb78 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:44:12.786502 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-185e4cf4-23d5-416f-825e-89cb852bcb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:12.796373 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:12.938189 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e4cf4-23d5-416f-825e-89cb852bcb78 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:44:13.020633 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:13.054367 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99f1fd-3d34-42d2-9734-46403e7b44e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb80229c3234d5f9599b16870c69934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:44:13.054968 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99f1fd-3d34-42d2-9734-46403e7b44e2 service nova] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:13.055270 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99f1fd-3d34-42d2-9734-46403e7b44e2 service nova] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:13.055537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99f1fd-3d34-42d2-9734-46403e7b44e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:13.057122 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-5c99f1fd-3d34-42d2-9734-46403e7b44e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:13.154724 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99f1fd-3d34-42d2-9734-46403e7b44e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:44:13.156349 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99f1fd-3d34-42d2-9734-46403e7b44e2 service nova] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:44:13.174501 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 760/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:44:15.223836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d3cd4-9f06-4450-8267-afe5814a7a62 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:15.224395 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d3cd4-9f06-4450-8267-afe5814a7a62 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:15.224395 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d3cd4-9f06-4450-8267-afe5814a7a62 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:15.226360 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 722/1482] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:44:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:44:15.231939 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189db8a0-dc14-4954-847e-7511073e774f None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:15.232337 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189db8a0-dc14-4954-847e-7511073e774f None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:15.232509 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189db8a0-dc14-4954-847e-7511073e774f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:15.301873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-189db8a0-dc14-4954-847e-7511073e774f None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:44:15.303065 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-189db8a0-dc14-4954-847e-7511073e774f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:15.351092 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:15.404720 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 761/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:44:15.692903 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d3cd4-9f06-4450-8267-afe5814a7a62 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:44:15.695405 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-0b1d3cd4-9f06-4450-8267-afe5814a7a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:15.726613 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:15.807233 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d3cd4-9f06-4450-8267-afe5814a7a62 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:44:15.881756 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c469f-38f1-4762-a58e-1f4a84532fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:44:15.882441 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c469f-38f1-4762-a58e-1f4a84532fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:15.882710 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c469f-38f1-4762-a58e-1f4a84532fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:15.882975 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c469f-38f1-4762-a58e-1f4a84532fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:15.884614 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-543c469f-38f1-4762-a58e-1f4a84532fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:15.960570 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:15.998141 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c469f-38f1-4762-a58e-1f4a84532fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:44:16.000040 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c469f-38f1-4762-a58e-1f4a84532fe6 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:44:16.025924 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 723/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:44:17.958707 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b061d7a-b1df-4315-81ba-ce2342be48c4 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:17.959362 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b061d7a-b1df-4315-81ba-ce2342be48c4 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:17.959362 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b061d7a-b1df-4315-81ba-ce2342be48c4 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:17.960723 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 762/1485] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:44:17.967045 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df918e-61d0-4d21-9c43-4c7971b75adc None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:17.967860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df918e-61d0-4d21-9c43-4c7971b75adc None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:17.967860 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df918e-61d0-4d21-9c43-4c7971b75adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:18.057349 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-74df918e-61d0-4d21-9c43-4c7971b75adc None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:44:18.058573 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-74df918e-61d0-4d21-9c43-4c7971b75adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:18.161882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 724/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:44:18.451758 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b061d7a-b1df-4315-81ba-ce2342be48c4 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:44:18.453293 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-8b061d7a-b1df-4315-81ba-ce2342be48c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:18.559873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b061d7a-b1df-4315-81ba-ce2342be48c4 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:44:18.636941 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:18.638825 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c40e9-c6c2-4e4c-859d-f98e0687a453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:44:18.639570 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c40e9-c6c2-4e4c-859d-f98e0687a453 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:18.639906 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c40e9-c6c2-4e4c-859d-f98e0687a453 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:18.640179 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c40e9-c6c2-4e4c-859d-f98e0687a453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:18.642276 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-5d5c40e9-c6c2-4e4c-859d-f98e0687a453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:18.762527 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c40e9-c6c2-4e4c-859d-f98e0687a453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:44:18.764021 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c40e9-c6c2-4e4c-859d-f98e0687a453 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:44:18.783661 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 763/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:44:20.762248 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c8af-2e8d-4a91-ae2f-3f9962d8a127 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:20.762836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c8af-2e8d-4a91-ae2f-3f9962d8a127 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:20.762836 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c8af-2e8d-4a91-ae2f-3f9962d8a127 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:20.764926 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 725/1488] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:44:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:44:20.773487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e88a0-2148-4801-97b9-fad0c49347e8 None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:20.773895 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e88a0-2148-4801-97b9-fad0c49347e8 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:20.774027 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e88a0-2148-4801-97b9-fad0c49347e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:20.842393 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-d44e88a0-2148-4801-97b9-fad0c49347e8 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:44:20.843612 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-d44e88a0-2148-4801-97b9-fad0c49347e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:20.907949 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:20.956332 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 764/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:44:21.249760 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29c8af-2e8d-4a91-ae2f-3f9962d8a127 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:44:21.252266 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-6f29c8af-2e8d-4a91-ae2f-3f9962d8a127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:21.302294 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:21.374824 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29c8af-2e8d-4a91-ae2f-3f9962d8a127 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:44:21.452338 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956680d2-dba4-4a3c-90c5-bd7e593f2157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:44:21.452954 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956680d2-dba4-4a3c-90c5-bd7e593f2157 service neutron] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:21.453227 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956680d2-dba4-4a3c-90c5-bd7e593f2157 service neutron] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:21.453494 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956680d2-dba4-4a3c-90c5-bd7e593f2157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:21.455211 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-956680d2-dba4-4a3c-90c5-bd7e593f2157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:21.540351 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:21.573482 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956680d2-dba4-4a3c-90c5-bd7e593f2157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:44:21.576196 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956680d2-dba4-4a3c-90c5-bd7e593f2157 service neutron] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:44:21.598320 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 726/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:44:23.766313 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85923aa9-eb72-4863-8342-cd6d91005fa6 None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:23.767140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85923aa9-eb72-4863-8342-cd6d91005fa6 None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:23.767140 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85923aa9-eb72-4863-8342-cd6d91005fa6 None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:23.769388 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 765/1491] 10.222.0.16 () {54 vars in 927 bytes} [Tue Aug 30 11:44:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:44:23.774928 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9d12c-eb26-4b41-9b9f-f8e89256d2ad None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:23.775402 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9d12c-eb26-4b41-9b9f-f8e89256d2ad None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:23.775518 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9d12c-eb26-4b41-9b9f-f8e89256d2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:23.850437 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-f0e9d12c-eb26-4b41-9b9f-f8e89256d2ad None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:44:23.851685 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-f0e9d12c-eb26-4b41-9b9f-f8e89256d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:23.952204 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 727/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Tue Aug 30 11:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:44:24.245393 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85923aa9-eb72-4863-8342-cd6d91005fa6 None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:44:24.247937 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-85923aa9-eb72-4863-8342-cd6d91005fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:24.363570 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85923aa9-eb72-4863-8342-cd6d91005fa6 None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:44:24.450931 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f342f1-9f0c-4cd4-bffb-0c5b73c8e338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:44:24.451592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f342f1-9f0c-4cd4-bffb-0c5b73c8e338 service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:44:24.451856 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f342f1-9f0c-4cd4-bffb-0c5b73c8e338 service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:44:24.452114 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f342f1-9f0c-4cd4-bffb-0c5b73c8e338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:44:24.453694 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-69f342f1-9f0c-4cd4-bffb-0c5b73c8e338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:44:24.464436 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:24.578404 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f342f1-9f0c-4cd4-bffb-0c5b73c8e338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:44:24.579818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f342f1-9f0c-4cd4-bffb-0c5b73c8e338 service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:44:24.605582 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 766/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:44:33.950363 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=82 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:44:34.601522 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=50 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:43.146484 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3c0f4-33a0-47bf-9ca8-2dd5a2237787 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:43.147104 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3c0f4-33a0-47bf-9ca8-2dd5a2237787 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:43.147104 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3c0f4-33a0-47bf-9ca8-2dd5a2237787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:43.235655 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-35f3c0f4-33a0-47bf-9ca8-2dd5a2237787 None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:45:43.236882 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-35f3c0f4-33a0-47bf-9ca8-2dd5a2237787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:43.336280 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 728/1494] 10.222.0.16 () {58 vars in 958 bytes} [Tue Aug 30 11:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:45:43.344722 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a827572-7f8e-45e2-ac01-f876b7700e2e None None] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:43.345233 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a827572-7f8e-45e2-ac01-f876b7700e2e None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:43.345233 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a827572-7f8e-45e2-ac01-f876b7700e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:43.423528 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.auth.core [None req-4a827572-7f8e-45e2-ac01-f876b7700e2e None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:45:43.424721 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-4a827572-7f8e-45e2-ac01-f876b7700e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:43.525858 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 767/1495] 10.222.0.16 () {58 vars in 958 bytes} [Tue Aug 30 11:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:45:43.530728 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3c0f4-33a0-47bf-9ca8-2dd5a2237787 None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:43.531990 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-35f3c0f4-33a0-47bf-9ca8-2dd5a2237787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:43.543651 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3c0f4-33a0-47bf-9ca8-2dd5a2237787 None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:43.615447 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:43.650658 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:43.651340 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:43.651610 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:43.651870 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] PATH_INFO: `/v3/domains` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:43.652510 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87979) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 11:45:43.652806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:43.653827 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: WARNING py.warnings [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 11:45:43.653827 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: warnings.warn(msg) Aug 30 11:45:43.653827 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: Aug 30 11:45:43.654719 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:43.666158 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 729/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Tue Aug 30 11:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 11:45:43.674779 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a827572-7f8e-45e2-ac01-f876b7700e2e None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:43.684258 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a827572-7f8e-45e2-ac01-f876b7700e2e None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:43.748732 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:43.784102 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:43.784740 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] REQUEST_METHOD: `POST` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:43.785008 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:43.785267 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] PATH_INFO: `/v3/projects` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:43.785916 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:43.787742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: WARNING py.warnings [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 11:45:43.787742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: warnings.warn(msg) Aug 30 11:45:43.787742 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: Aug 30 11:45:43.788944 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:43.851876 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 768/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Tue Aug 30 11:45:43 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:45:43.856299 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:43.857837 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:43.890689 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363af1a-65dc-4900-b770-9d93ec060c3e admin admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:43.988065 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:43.988666 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:43.988940 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:43.989207 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] PATH_INFO: `/v3/users` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:43.989852 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:43.990891 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: WARNING py.warnings [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 11:45:43.990891 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: warnings.warn(msg) Aug 30 11:45:43.990891 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: Aug 30 11:45:43.992142 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:44.035830 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 730/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Aug 30 11:45:43 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 11:45:44.040539 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:44.042082 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:44.109180 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f56104-846f-482b-a726-6b7b3a8b492e admin admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:44.197184 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:44.209872 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:44.210475 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:44.210756 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:44.211017 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:44.211728 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:44.212739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: WARNING py.warnings [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 11:45:44.212739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: warnings.warn(msg) Aug 30 11:45:44.212739 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: Aug 30 11:45:44.213625 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:44.226494 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 769/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Tue Aug 30 11:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 11:45:44.230917 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:44.232466 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:44.244658 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ef20f-24b4-4793-92e1-c85df78322bb admin admin] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:44.270884 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:44.343365 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cc8c5-9934-43fb-944f-2030048bc194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:44.343962 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cc8c5-9934-43fb-944f-2030048bc194 admin admin] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:44.344239 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cc8c5-9934-43fb-944f-2030048bc194 admin admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:44.344502 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cc8c5-9934-43fb-944f-2030048bc194 admin admin] PATH_INFO: `/v3/roles` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:44.345150 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cc8c5-9934-43fb-944f-2030048bc194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:44.346153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: WARNING py.warnings [None req-7a4cc8c5-9934-43fb-944f-2030048bc194 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 11:45:44.346153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: warnings.warn(msg) Aug 30 11:45:44.346153 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: Aug 30 11:45:44.347077 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cc8c5-9934-43fb-944f-2030048bc194 admin admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:44.365966 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 731/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Tue Aug 30 11:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 11:45:44.370617 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:44.372197 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:44.407558 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b47df7-b2ed-4e29-ba90-981b1f776b00 admin admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:44.513007 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7373e62-2b29-4c2c-94a3-0ee91d522077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:44.513594 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7373e62-2b29-4c2c-94a3-0ee91d522077 admin admin] REQUEST_METHOD: `PUT` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:44.513873 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7373e62-2b29-4c2c-94a3-0ee91d522077 admin admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:44.514128 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7373e62-2b29-4c2c-94a3-0ee91d522077 admin admin] PATH_INFO: `/v3/projects/21eff398fdc549129d4f18f268309308/users/c18e2836cf2a4ad8acc5f3fc52a59f9c/roles/91e10f3d9d934fe1be9d3642b8092d08` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:44.538122 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7373e62-2b29-4c2c-94a3-0ee91d522077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21eff398fdc549129d4f18f268309308, user_id=c18e2836cf2a4ad8acc5f3fc52a59f9c, role_id=91e10f3d9d934fe1be9d3642b8092d08)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:44.539153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: WARNING py.warnings [None req-a7373e62-2b29-4c2c-94a3-0ee91d522077 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 11:45:44.539153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: warnings.warn(msg) Aug 30 11:45:44.539153 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: Aug 30 11:45:44.541145 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7373e62-2b29-4c2c-94a3-0ee91d522077 admin admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:44.567507 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 770/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Tue Aug 30 11:45:44 2022] PUT /identity/v3/projects/21eff398fdc549129d4f18f268309308/users/c18e2836cf2a4ad8acc5f3fc52a59f9c/roles/91e10f3d9d934fe1be9d3642b8092d08 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:45:44.580717 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130d35-e177-4a88-8be3-91da4c46b985 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:44.581000 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130d35-e177-4a88-8be3-91da4c46b985 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:44.581300 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130d35-e177-4a88-8be3-91da4c46b985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:44.648475 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-a3130d35-e177-4a88-8be3-91da4c46b985 None None] MFA Rules not processed for user `c18e2836cf2a4ad8acc5f3fc52a59f9c`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:45:44.649706 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-a3130d35-e177-4a88-8be3-91da4c46b985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:44.733553 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 732/1502] 10.222.0.16 () {58 vars in 958 bytes} [Tue Aug 30 11:45:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:45:44.752005 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1820d89-7f6c-4e42-a641-879f1a5b959c None None] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:44.752285 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1820d89-7f6c-4e42-a641-879f1a5b959c None None] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:44.752551 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1820d89-7f6c-4e42-a641-879f1a5b959c None None] PATH_INFO: `/` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:44.753930 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 771/1503] 10.222.0.16 () {54 vars in 964 bytes} [Tue Aug 30 11:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:45:44.758944 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c4f4-ce23-4aa8-a65b-42de1d8aec70 None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:44.759251 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c4f4-ce23-4aa8-a65b-42de1d8aec70 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:44.759516 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c4f4-ce23-4aa8-a65b-42de1d8aec70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:44.917839 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:44.963761 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-63a1c4f4-ce23-4aa8-a65b-42de1d8aec70 None None] MFA Rules not processed for user `d3250e8f23c64f26b4e862090f7f2e18`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:45:44.964979 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-63a1c4f4-ce23-4aa8-a65b-42de1d8aec70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:45.050499 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 733/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Tue Aug 30 11:45:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:45:45.060602 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1820d89-7f6c-4e42-a641-879f1a5b959c None None] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:45.062122 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-f1820d89-7f6c-4e42-a641-879f1a5b959c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:45.116419 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:45.171803 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1820d89-7f6c-4e42-a641-879f1a5b959c None None] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:45.255723 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3250e8f23c64f26b4e862090f7f2e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:45.256336 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:45.256601 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:45.256888 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:45.258490 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:45.360504 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:45.361946 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:45.365461 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:45.382071 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 772/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 11:45:45.426462 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b09d0-84d2-4e1e-bedb-138a7edcc6c5 None None] REQUEST_METHOD: `GET` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:45.426912 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b09d0-84d2-4e1e-bedb-138a7edcc6c5 None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:45.426912 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b09d0-84d2-4e1e-bedb-138a7edcc6c5 None None] PATH_INFO: `/` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:45.428246 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 734/1506] 10.222.0.16 () {52 vars in 830 bytes} [Tue Aug 30 11:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 11:45:45.431940 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:45.433176 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:45.539741 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51682eaa-10a6-4953-8a07-f8ca323ea112 service cinder] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:45.639725 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:45.640342 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:45.640606 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:45.640868 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] PATH_INFO: `/v3/users/c18e2836cf2a4ad8acc5f3fc52a59f9c` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:45.654973 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18e2836cf2a4ad8acc5f3fc52a59f9c)` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:45.656043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: WARNING py.warnings [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 11:45:45.656043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: warnings.warn(msg) Aug 30 11:45:45.656043 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: Aug 30 11:45:45.657288 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:45.667716 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=100 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:45.727074 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18e2836cf2a4ad8acc5f3fc52a59f9c'} {{(pid=87980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:45.727537 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18e2836cf2a4ad8acc5f3fc52a59f9c'} {{(pid=87980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:45.755166 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18e2836cf2a4ad8acc5f3fc52a59f9c'} {{(pid=87980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:45.775028 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18e2836cf2a4ad8acc5f3fc52a59f9c'} {{(pid=87980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:45.775703 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.notifications [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18e2836cf2a4ad8acc5f3fc52a59f9c'} {{(pid=87980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:45.792487 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 773/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Tue Aug 30 11:45:45 2022] DELETE /identity/v3/users/c18e2836cf2a4ad8acc5f3fc52a59f9c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:45:45.797085 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174582d-5362-4320-8203-4443908f25ce None None] REQUEST_METHOD: `POST` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:45.797387 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174582d-5362-4320-8203-4443908f25ce None None] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:45.797662 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174582d-5362-4320-8203-4443908f25ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:45.856991 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.auth.core [None req-0174582d-5362-4320-8203-4443908f25ce None None] MFA Rules not processed for user `506323730e1e4e22bdf37b533b4f458b`. Rule list: `[]` (Enabled: `True`). {{(pid=87979) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 11:45:45.858219 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-0174582d-5362-4320-8203-4443908f25ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:45.936898 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=100 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:45.951719 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 735/1508] 10.222.0.16 () {58 vars in 958 bytes} [Tue Aug 30 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 11:45:45.961598 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] Authenticating user token {{(pid=87980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:45.962859 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:46.073332 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3047b7-08de-4eec-bf45-700ee7e4ce00 admin admin] Validating token access rules against request {{(pid=87980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:46.130851 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:46.151627 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a3532-0694-4575-81a7-d3156222b10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56bc4fdd14d147ef86b96786da2855f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0027f77ae674bc08f99838ef869845c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:46.152271 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a3532-0694-4575-81a7-d3156222b10b service neutron] REQUEST_METHOD: `GET` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:46.152560 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a3532-0694-4575-81a7-d3156222b10b service neutron] SCRIPT_NAME: `/identity` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:46.152818 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a3532-0694-4575-81a7-d3156222b10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:46.154434 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.fernet_utils [None req-838a3532-0694-4575-81a7-d3156222b10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:46.407669 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a3532-0694-4575-81a7-d3156222b10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:46.409058 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a3532-0694-4575-81a7-d3156222b10b service neutron] RBAC: Authorization granted {{(pid=87980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:46.428026 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: [pid: 87980|app: 0|req: 774/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Aug 30 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 11:45:46.776717 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174582d-5362-4320-8203-4443908f25ce None None] Authenticating user token {{(pid=87979) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 11:45:46.777965 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.fernet_utils [None req-0174582d-5362-4320-8203-4443908f25ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87979) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 11:45:46.789633 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174582d-5362-4320-8203-4443908f25ce None None] Validating token access rules against request {{(pid=87979) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 11:45:46.890111 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '506323730e1e4e22bdf37b533b4f458b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2579099690b4b9ea0cc6cb7695b8b5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87979) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 11:45:46.890743 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 11:45:46.891014 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] SCRIPT_NAME: `/identity` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 11:45:46.891304 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] PATH_INFO: `/v3/projects/21eff398fdc549129d4f18f268309308` {{(pid=87979) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 11:45:46.901786 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] RBAC: Authorizing `identity:delete_project(project_id=21eff398fdc549129d4f18f268309308)` {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 11:45:46.902799 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: WARNING py.warnings [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 11:45:46.902799 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: warnings.warn(msg) Aug 30 11:45:46.902799 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: Aug 30 11:45:46.904060 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] RBAC: Authorization granted {{(pid=87979) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 11:45:46.965336 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21eff398fdc549129d4f18f268309308', 'request_id': 'req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a'} {{(pid=87979) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:46.976016 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21eff398fdc549129d4f18f268309308', 'request_id': 'req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a'} {{(pid=87979) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:46.989758 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21eff398fdc549129d4f18f268309308', 'request_id': 'req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a'} {{(pid=87979) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:47.012806 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] The token cache is being invalidate because project 21eff398fdc549129d4f18f268309308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87979) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 11:45:47.013086 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87979) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:47.013597 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG keystone.notifications [None req-ae5bc8ce-7b32-4a7d-b0de-e56b5aed214a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87979) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 11:45:47.014907 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: [pid: 87979|app: 0|req: 736/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Tue Aug 30 11:45:46 2022] DELETE /identity/v3/projects/21eff398fdc549129d4f18f268309308 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 11:45:56.425592 dv-cinder-852208-12-iscsi devstack@keystone.service[87980]: DEBUG dbcounter [-] [87980] Writing DB stats keystone:SELECT=62 {{(pid=87980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 11:45:57.010794 dv-cinder-852208-12-iscsi devstack@keystone.service[87979]: DEBUG dbcounter [-] [87979] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87979) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}